pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feed[PATCH 4/5] Shared-memory based stats collector
17+ messages / 5 participants
[nested] [flat]
* [PATCH 4/5] Shared-memory based stats collector
@ 2018-11-12 08:26 Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2018-11-12 08:26 UTC (permalink / raw)
Previously activity statistics is shared via files on disk. Every
backend sends the numbers to the stats collector process via a socket.
It makes snapshots as a set of files on disk with a certain interval
then every backend reads them as necessary. It worked fine for
comparatively small set of statistics but the set is under the
pressure to growing up and the file size has reached the order of
megabytes. To deal with larger statistics set, this patch let backends
directly share the statistics via shared memory.
---
contrib/pg_prewarm/autoprewarm.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 1 +
contrib/postgres_fdw/connection.c | 2 +-
src/backend/Makefile | 2 +-
src/backend/access/heap/rewriteheap.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 1 +
src/backend/access/nbtree/nbtree.c | 2 +-
src/backend/access/nbtree/nbtsort.c | 2 +-
src/backend/access/transam/clog.c | 2 +-
src/backend/access/transam/parallel.c | 1 +
src/backend/access/transam/slru.c | 2 +-
src/backend/access/transam/timeline.c | 2 +-
src/backend/access/transam/twophase.c | 2 +
src/backend/access/transam/xact.c | 3 +
src/backend/access/transam/xlog.c | 5 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/access/transam/xlogutils.c | 2 +-
src/backend/bootstrap/bootstrap.c | 8 +-
src/backend/executor/execParallel.c | 2 +-
src/backend/executor/nodeBitmapHeapscan.c | 1 +
src/backend/executor/nodeGather.c | 1 +
src/backend/executor/nodeHash.c | 2 +-
src/backend/executor/nodeHashjoin.c | 2 +-
src/backend/libpq/be-secure-openssl.c | 2 +-
src/backend/libpq/be-secure.c | 2 +-
src/backend/libpq/pqmq.c | 2 +-
src/backend/postmaster/Makefile | 2 +-
src/backend/postmaster/autovacuum.c | 46 +-
src/backend/postmaster/bgworker.c | 2 +-
src/backend/postmaster/bgwriter.c | 5 +-
src/backend/postmaster/checkpointer.c | 17 +-
src/backend/postmaster/pgarch.c | 5 +-
src/backend/postmaster/pgstat.c | 6385 --------------------
src/backend/postmaster/postmaster.c | 86 +-
src/backend/postmaster/syslogger.c | 2 +-
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/basebackup.c | 1 +
.../libpqwalreceiver/libpqwalreceiver.c | 2 +-
src/backend/replication/logical/launcher.c | 2 +-
src/backend/replication/logical/origin.c | 3 +-
src/backend/replication/logical/reorderbuffer.c | 2 +-
src/backend/replication/logical/snapbuild.c | 2 +-
src/backend/replication/logical/tablesync.c | 15 +-
src/backend/replication/logical/worker.c | 5 +-
src/backend/replication/slot.c | 2 +-
src/backend/replication/syncrep.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/backend/replication/walsender.c | 2 +-
src/backend/statmon/Makefile | 17 +
src/backend/statmon/bestatus.c | 1781 ++++++
src/backend/statmon/pgstat.c | 3962 ++++++++++++
src/backend/storage/buffer/bufmgr.c | 1 +
src/backend/storage/file/buffile.c | 2 +-
src/backend/storage/file/copydir.c | 2 +-
src/backend/storage/file/fd.c | 1 +
src/backend/storage/ipc/dsm.c | 24 +-
src/backend/storage/ipc/dsm_impl.c | 2 +-
src/backend/storage/ipc/ipci.c | 6 +
src/backend/storage/ipc/latch.c | 2 +-
src/backend/storage/ipc/procarray.c | 2 +-
src/backend/storage/ipc/shm_mq.c | 2 +-
src/backend/storage/ipc/standby.c | 2 +-
src/backend/storage/lmgr/deadlock.c | 1 +
src/backend/storage/lmgr/lwlock.c | 5 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/lmgr/predicate.c | 2 +-
src/backend/storage/lmgr/proc.c | 2 +-
src/backend/storage/smgr/md.c | 2 +-
src/backend/tcop/postgres.c | 28 +-
src/backend/utils/adt/misc.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 51 +-
src/backend/utils/cache/relmapper.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 2 +-
src/backend/utils/init/postinit.c | 15 +
src/backend/utils/misc/guc.c | 1 +
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/include/bestatus.h | 555 ++
src/include/miscadmin.h | 2 +-
src/include/pgstat.h | 937 +--
src/include/storage/dsm.h | 3 +
src/include/storage/lwlock.h | 3 +
src/include/utils/timeout.h | 1 +
src/test/modules/worker_spi/worker_spi.c | 2 +-
84 files changed, 6597 insertions(+), 7480 deletions(-)
delete mode 100644 src/backend/postmaster/pgstat.c
create mode 100644 src/backend/statmon/Makefile
create mode 100644 src/backend/statmon/bestatus.c
create mode 100644 src/backend/statmon/pgstat.c
create mode 100644 src/include/bestatus.h
inline patch 1: contrib/pg_prewarm/autoprewarm.c, contrib/pg_stat_statements/pg_stat_statements.c, contrib/postgres_fdw/connection.c and 81 more (+6597/-7480 lines) download inline-1.patch
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index 9cc4b2dc83..406efbd49b 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -30,10 +30,10 @@
#include "access/relation.h"
#include "access/xact.h"
+#include "bestatus.h"
#include "catalog/pg_class.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "storage/buf_internals.h"
#include "storage/dsm.h"
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index ea2e4bc242..7084960742 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -62,6 +62,7 @@
#include <unistd.h>
#include "access/hash.h"
+#include "bestatus.h"
#include "catalog/pg_authid.h"
#include "executor/instrument.h"
#include "funcapi.h"
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 239d220c24..1ea71245df 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -15,11 +15,11 @@
#include "postgres_fdw.h"
#include "access/htup_details.h"
+#include "bestatus.h"
#include "catalog/pg_user_mapping.h"
#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/latch.h"
#include "utils/hsearch.h"
#include "utils/inval.h"
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 478a96db9b..cc511672c9 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -20,7 +20,7 @@ include $(top_builddir)/src/Makefile.global
SUBDIRS = access bootstrap catalog parser commands executor foreign lib libpq \
main nodes optimizer partitioning port postmaster \
regex replication rewrite \
- statistics storage tcop tsearch utils $(top_builddir)/src/timezone \
+ statistics statmon storage tcop tsearch utils $(top_builddir)/src/timezone \
jit
include $(srcdir)/common.mk
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index f5cf9ffc9c..adfd5f40fd 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -115,12 +115,12 @@
#include "access/xact.h"
#include "access/xloginsert.h"
+#include "bestatus.h"
+
#include "catalog/catalog.h"
#include "lib/ilist.h"
-#include "pgstat.h"
-
#include "replication/logical.h"
#include "replication/slot.h"
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 9416c31889..928d53a68c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -44,6 +44,7 @@
#include "access/transam.h"
#include "access/visibilitymap.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "catalog/storage.h"
#include "commands/dbcommands.h"
#include "commands/progress.h"
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index 98917de2ef..69cd211369 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -22,10 +22,10 @@
#include "access/nbtxlog.h"
#include "access/relscan.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "commands/vacuum.h"
#include "miscadmin.h"
#include "nodes/execnodes.h"
-#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "storage/condition_variable.h"
#include "storage/indexfsm.h"
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index dc398e1186..c2a3ed0209 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -64,9 +64,9 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "access/xloginsert.h"
+#include "bestatus.h"
#include "catalog/index.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/smgr.h"
#include "tcop/tcopprot.h" /* pgrminclude ignore */
#include "utils/rel.h"
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index aa089d83fa..cf034ba333 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -38,8 +38,8 @@
#include "access/xlog.h"
#include "access/xloginsert.h"
#include "access/xlogutils.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "pg_trace.h"
#include "storage/proc.h"
diff --git a/src/backend/access/transam/parallel.c b/src/backend/access/transam/parallel.c
index ce2b61631d..8d5cbfa41d 100644
--- a/src/backend/access/transam/parallel.c
+++ b/src/backend/access/transam/parallel.c
@@ -19,6 +19,7 @@
#include "access/session.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "catalog/pg_enum.h"
#include "catalog/index.h"
#include "catalog/namespace.h"
diff --git a/src/backend/access/transam/slru.c b/src/backend/access/transam/slru.c
index 3623352b9c..a28fe474aa 100644
--- a/src/backend/access/transam/slru.c
+++ b/src/backend/access/transam/slru.c
@@ -54,7 +54,7 @@
#include "access/slru.h"
#include "access/transam.h"
#include "access/xlog.h"
-#include "pgstat.h"
+#include "bestatus.h"
#include "storage/fd.h"
#include "storage/shmem.h"
#include "miscadmin.h"
diff --git a/src/backend/access/transam/timeline.c b/src/backend/access/transam/timeline.c
index c96c8b60ba..bbe9c0eb5f 100644
--- a/src/backend/access/transam/timeline.c
+++ b/src/backend/access/transam/timeline.c
@@ -38,7 +38,7 @@
#include "access/xlog.h"
#include "access/xlog_internal.h"
#include "access/xlogdefs.h"
-#include "pgstat.h"
+#include "bestatus.h"
#include "storage/fd.h"
/*
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9a8a6bb119..0dc9f39424 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -87,6 +87,7 @@
#include "access/xloginsert.h"
#include "access/xlogutils.h"
#include "access/xlogreader.h"
+#include "bestatus.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
#include "funcapi.h"
@@ -1569,6 +1570,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
PredicateLockTwoPhaseFinish(xid, isCommit);
/* Count the prepared xact as committed or aborted */
+ AtEOXact_BEStatus(isCommit);
AtEOXact_PgStat(isCommit);
/*
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 92bda87804..ac32c44e05 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -30,6 +30,7 @@
#include "access/xlog.h"
#include "access/xloginsert.h"
#include "access/xlogutils.h"
+#include "bestatus.h"
#include "catalog/namespace.h"
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
@@ -2148,6 +2149,7 @@ CommitTransaction(void)
AtEOXact_Files(true);
AtEOXact_ComboCid();
AtEOXact_HashTables(true);
+ AtEOXact_BEStatus(true);
AtEOXact_PgStat(true);
AtEOXact_Snapshot(true, false);
AtEOXact_ApplyLauncher(true);
@@ -2641,6 +2643,7 @@ AbortTransaction(void)
AtEOXact_Files(false);
AtEOXact_ComboCid();
AtEOXact_HashTables(false);
+ AtEOXact_BEStatus(false);
AtEOXact_PgStat(false);
AtEOXact_ApplyLauncher(false);
pgstat_report_xact_timestamp(0);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ecd12fc53a..c0f0a7195c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -36,6 +36,7 @@
#include "access/xloginsert.h"
#include "access/xlogreader.h"
#include "access/xlogutils.h"
+#include "bestatus.h"
#include "catalog/catversion.h"
#include "catalog/pg_control.h"
#include "catalog/pg_database.h"
@@ -8420,9 +8421,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index b35043bf71..683c41575f 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -23,9 +23,9 @@
#include "access/xlog_internal.h"
#include "access/xlogutils.h"
#include "catalog/pg_type.h"
+#include "bestatus.h"
#include "funcapi.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "replication/walreceiver.h"
#include "storage/smgr.h"
#include "utils/builtins.h"
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index 10a663bae6..53fa4890e9 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -23,8 +23,8 @@
#include "access/xlog.h"
#include "access/xlog_internal.h"
#include "access/xlogutils.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/smgr.h"
#include "utils/guc.h"
#include "utils/hsearch.h"
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index a6c3338d40..79f624f0e0 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "bootstrap/bootstrap.h"
#include "catalog/index.h"
#include "catalog/pg_collation.h"
@@ -328,9 +329,6 @@ AuxiliaryProcessMain(int argc, char *argv[])
case BgWriterProcess:
statmsg = pgstat_get_backend_desc(B_BG_WRITER);
break;
- case ArchiverProcess:
- statmsg = pgstat_get_backend_desc(B_ARCHIVER);
- break;
case CheckpointerProcess:
statmsg = pgstat_get_backend_desc(B_CHECKPOINTER);
break;
@@ -340,6 +338,9 @@ AuxiliaryProcessMain(int argc, char *argv[])
case WalReceiverProcess:
statmsg = pgstat_get_backend_desc(B_WAL_RECEIVER);
break;
+ case ArchiverProcess:
+ statmsg = pgstat_get_backend_desc(B_ARCHIVER);
+ break;
default:
statmsg = "??? process";
break;
@@ -416,6 +417,7 @@ AuxiliaryProcessMain(int argc, char *argv[])
CreateAuxProcessResourceOwner();
/* Initialize backend status information */
+ pgstat_bearray_initialize();
pgstat_initialize();
pgstat_bestart();
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index b79be91655..e53c0fb808 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -46,7 +46,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
-#include "pgstat.h"
+#include "bestatus.h"
/*
* Magic numbers for parallel executor communication. We use constants
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5e74585d5e..03a703075e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -41,6 +41,7 @@
#include "access/relscan.h"
#include "access/transam.h"
#include "access/visibilitymap.h"
+#include "bestatus.h"
#include "executor/execdebug.h"
#include "executor/nodeBitmapHeapscan.h"
#include "miscadmin.h"
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 69d5a1f239..36859360b6 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -32,6 +32,7 @@
#include "access/relscan.h"
#include "access/xact.h"
+#include "bestatus.h"
#include "executor/execdebug.h"
#include "executor/execParallel.h"
#include "executor/nodeGather.h"
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 856daf6a7f..5a47eb4601 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -28,6 +28,7 @@
#include "access/htup_details.h"
#include "access/parallel.h"
+#include "bestatus.h"
#include "catalog/pg_statistic.h"
#include "commands/tablespace.h"
#include "executor/execdebug.h"
@@ -35,7 +36,6 @@
#include "executor/nodeHash.h"
#include "executor/nodeHashjoin.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "port/atomics.h"
#include "utils/dynahash.h"
#include "utils/memutils.h"
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 2098708864..898a7916b0 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -108,12 +108,12 @@
#include "access/htup_details.h"
#include "access/parallel.h"
+#include "bestatus.h"
#include "executor/executor.h"
#include "executor/hashjoin.h"
#include "executor/nodeHash.h"
#include "executor/nodeHashjoin.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "utils/memutils.h"
#include "utils/sharedtuplestore.h"
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index d1417454f2..4eead99293 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -36,9 +36,9 @@
#include <openssl/ec.h>
#endif
+#include "bestatus.h"
#include "libpq/libpq.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/fd.h"
#include "storage/latch.h"
#include "tcop/tcopprot.h"
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index a7def3168d..fa1cf6cffa 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -29,9 +29,9 @@
#include <arpa/inet.h>
#endif
+#include "bestatus.h"
#include "libpq/libpq.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "tcop/tcopprot.h"
#include "utils/memutils.h"
#include "storage/ipc.h"
diff --git a/src/backend/libpq/pqmq.c b/src/backend/libpq/pqmq.c
index a9bd47d937..f79a70d6fe 100644
--- a/src/backend/libpq/pqmq.c
+++ b/src/backend/libpq/pqmq.c
@@ -13,11 +13,11 @@
#include "postgres.h"
+#include "bestatus.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "libpq/pqmq.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index 71c23211b2..311e63017d 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -13,6 +13,6 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = autovacuum.o bgworker.o bgwriter.o checkpointer.o fork_process.o \
- pgarch.o pgstat.o postmaster.o startup.o syslogger.o walwriter.o
+ pgarch.o postmaster.o startup.o syslogger.o walwriter.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index d1177b3855..b1328d34f5 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -71,6 +71,7 @@
#include "access/reloptions.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "bestatus.h"
#include "catalog/dependency.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
@@ -968,7 +969,7 @@ rebuild_database_list(Oid newdb)
PgStat_StatDBEntry *entry;
/* only consider this database if it has a pgstat entry */
- entry = pgstat_fetch_stat_dbentry(newdb);
+ entry = pgstat_fetch_stat_dbentry(newdb, true);
if (entry != NULL)
{
/* we assume it isn't found because the hash was just created */
@@ -977,6 +978,7 @@ rebuild_database_list(Oid newdb)
/* hash_search already filled in the key */
db->adl_score = score++;
/* next_worker is filled in later */
+ pfree(entry);
}
}
@@ -992,7 +994,7 @@ rebuild_database_list(Oid newdb)
* skip databases with no stat entries -- in particular, this gets rid
* of dropped databases
*/
- entry = pgstat_fetch_stat_dbentry(avdb->adl_datid);
+ entry = pgstat_fetch_stat_dbentry(avdb->adl_datid, true);
if (entry == NULL)
continue;
@@ -1004,6 +1006,7 @@ rebuild_database_list(Oid newdb)
db->adl_score = score++;
/* next_worker is filled in later */
}
+ pfree(entry);
}
/* finally, insert all qualifying databases not previously inserted */
@@ -1016,7 +1019,7 @@ rebuild_database_list(Oid newdb)
PgStat_StatDBEntry *entry;
/* only consider databases with a pgstat entry */
- entry = pgstat_fetch_stat_dbentry(avdb->adw_datid);
+ entry = pgstat_fetch_stat_dbentry(avdb->adw_datid, true);
if (entry == NULL)
continue;
@@ -1028,6 +1031,7 @@ rebuild_database_list(Oid newdb)
db->adl_score = score++;
/* next_worker is filled in later */
}
+ pfree(entry);
}
nelems = score;
@@ -1226,7 +1230,7 @@ do_start_worker(void)
continue; /* ignore not-at-risk DBs */
/* Find pgstat entry if any */
- tmp->adw_entry = pgstat_fetch_stat_dbentry(tmp->adw_datid);
+ tmp->adw_entry = pgstat_fetch_stat_dbentry(tmp->adw_datid, true);
/*
* Skip a database with no pgstat entry; it means it hasn't seen any
@@ -1265,7 +1269,12 @@ do_start_worker(void)
}
}
if (skipit)
+ {
+ /* Immediately free it if not used */
+ if(avdb != tmp)
+ pfree(tmp->adw_entry);
continue;
+ }
/*
* Remember the db with oldest autovac time. (If we are here, both
@@ -1273,7 +1282,12 @@ do_start_worker(void)
*/
if (avdb == NULL ||
tmp->adw_entry->last_autovac_time < avdb->adw_entry->last_autovac_time)
+ {
+ if (avdb)
+ pfree(avdb->adw_entry);
+
avdb = tmp;
+ }
}
/* Found a database -- process it */
@@ -1962,7 +1976,7 @@ do_autovacuum(void)
* may be NULL if we couldn't find an entry (only happens if we are
* forcing a vacuum for anti-wrap purposes).
*/
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId, true);
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
@@ -2012,7 +2026,7 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
/* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
+ shared = pgstat_fetch_stat_dbentry(InvalidOid, true);
classRel = table_open(RelationRelationId, AccessShareLock);
@@ -2098,6 +2112,8 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
+ if (tabentry)
+ pfree(tabentry);
/* Relations that need work are added to table_oids */
if (dovacuum || doanalyze)
@@ -2177,10 +2193,11 @@ do_autovacuum(void)
/* Fetch the pgstat entry for this table */
tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
shared, dbentry);
-
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
+ if (tabentry)
+ pfree(tabentry);
/* ignore analyze for toast tables */
if (dovacuum)
@@ -2749,12 +2766,10 @@ get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
if (isshared)
{
if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = backend_get_tab_entry(shared, relid, true);
}
else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = backend_get_tab_entry(dbentry, relid, true);
return tabentry;
}
@@ -2786,8 +2801,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
/* use fresh stats */
autovac_refresh_stats();
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
+ shared = pgstat_fetch_stat_dbentry(InvalidOid, true);
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId, true);
/* fetch the relation's relcache entry */
classTup = SearchSysCacheCopy1(RELOID, ObjectIdGetDatum(relid));
@@ -2818,6 +2833,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
+ if (tabentry)
+ pfree(tabentry);
/* ignore ANALYZE for toast tables */
if (classForm->relkind == RELKIND_TOASTVALUE)
@@ -2908,7 +2925,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
}
heap_freetuple(classTup);
-
+ pfree(shared);
+ pfree(dbentry);
return tab;
}
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index f5db5a8c4a..7d7d55ef1a 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -16,8 +16,8 @@
#include "libpq/pqsignal.h"
#include "access/parallel.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "port/atomics.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/postmaster.h"
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index e6b6c549de..c820d35fbc 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -40,6 +40,7 @@
#include "access/xlog.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -267,9 +268,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Update activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_update_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index fe96c41359..9f70cd0e52 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -43,6 +43,7 @@
#include "access/xlog.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -515,13 +516,13 @@ CheckpointerMain(void)
CheckArchiveTimeout();
/*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
+ * Update activity statistics. (The reason why we re-use
+ * bgwriter-related code for this is that the bgwriter and
+ * checkpointer used to be just one process. It's probably not worth
+ * the trouble to split the stats support into two independent
+ * functions.)
*/
- pgstat_send_bgwriter();
+ pgstat_update_bgwriter();
/*
* Sleep until we are signaled or it's time for another checkpoint or
@@ -682,9 +683,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Register interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_update_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 4342ebdab4..2a7c4fd1b1 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -35,6 +35,7 @@
#include "access/xlog.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -468,7 +469,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we successfully
* archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_update_archiver(xlog, false);
break; /* out of inner retry loop */
}
@@ -478,7 +479,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we failed to
* archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_update_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
deleted file mode 100644
index 9e6bce8f6a..0000000000
--- a/src/backend/postmaster/pgstat.c
+++ /dev/null
@@ -1,6385 +0,0 @@
-/* ----------
- * pgstat.c
- *
- * All the statistics collector stuff hacked up in one big, ugly file.
- *
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
- *
- * - Add some automatic call for pgstat vacuuming.
- *
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
- *
- * Copyright (c) 2001-2019, PostgreSQL Global Development Group
- *
- * src/backend/postmaster/pgstat.c
- * ----------
- */
-#include "postgres.h"
-
-#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
-
-#include "pgstat.h"
-
-#include "access/heapam.h"
-#include "access/htup_details.h"
-#include "access/transam.h"
-#include "access/twophase_rmgr.h"
-#include "access/xact.h"
-#include "catalog/pg_database.h"
-#include "catalog/pg_proc.h"
-#include "common/ip.h"
-#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
-#include "miscadmin.h"
-#include "pg_trace.h"
-#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
-#include "postmaster/postmaster.h"
-#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
-#include "storage/ipc.h"
-#include "storage/latch.h"
-#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
-#include "storage/procsignal.h"
-#include "storage/sinvaladt.h"
-#include "utils/ascii.h"
-#include "utils/guc.h"
-#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
-#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
-
-
-/* ----------
- * Timer definitions.
- * ----------
- */
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
-
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
-
-
-/* ----------
- * The initial size hints for the hash tables used in the collector.
- * ----------
- */
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
-#define PGSTAT_FUNCTION_HASH_SIZE 512
-
-
-/* ----------
- * Total number of backends including auxiliary
- *
- * We reserve a slot for each possible BackendId, plus one for each
- * possible auxiliary process type. (This scheme assumes there is not
- * more than one of any auxiliary process type at a time.) MaxBackends
- * includes autovacuum workers and background workers as well.
- * ----------
- */
-#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
-
-/* ----------
- * GUC parameters
- * ----------
- */
-bool pgstat_track_activities = false;
-bool pgstat_track_counts = false;
-int pgstat_track_functions = TRACK_FUNC_OFF;
-int pgstat_track_activity_query_size = 1024;
-
-/* ----------
- * Built from GUC parameter
- * ----------
- */
-char *pgstat_stat_directory = NULL;
-char *pgstat_stat_filename = NULL;
-char *pgstat_stat_tmpname = NULL;
-
-/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-
-/* ----------
- * Local data
- * ----------
- */
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
-
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
-
-/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
- *
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
- */
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-
-typedef struct TabStatusArray
-{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
-
-/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
- */
-typedef struct TabStatHashEntry
-{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
-
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
-
-/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
- */
-static HTAB *pgStatFunctions = NULL;
-
-/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
- */
-static bool have_function_stats = false;
-
-/*
- * Tuple insertion/deletion counts for an open transaction can't be propagated
- * into PgStat_TableStatus counters until we know if it is going to commit
- * or abort. Hence, we keep these counts in per-subxact structs that live
- * in TopTransactionContext. This data structure is designed on the assumption
- * that subxacts won't usually modify very many tables.
- */
-typedef struct PgStat_SubXactStatus
-{
- int nest_level; /* subtransaction nest level */
- struct PgStat_SubXactStatus *prev; /* higher-level subxact if any */
- PgStat_TableXactStatus *first; /* head of list for this subxact */
-} PgStat_SubXactStatus;
-
-static PgStat_SubXactStatus *pgStatXactStack = NULL;
-
-static int pgStatXactCommit = 0;
-static int pgStatXactRollback = 0;
-PgStat_Counter pgStatBlockReadTime = 0;
-PgStat_Counter pgStatBlockWriteTime = 0;
-
-/* Record that's written to 2PC state file when pgstat state is persisted */
-typedef struct TwoPhasePgStatRecord
-{
- PgStat_Counter tuples_inserted; /* tuples inserted in xact */
- PgStat_Counter tuples_updated; /* tuples updated in xact */
- PgStat_Counter tuples_deleted; /* tuples deleted in xact */
- PgStat_Counter inserted_pre_trunc; /* tuples inserted prior to truncate */
- PgStat_Counter updated_pre_trunc; /* tuples updated prior to truncate */
- PgStat_Counter deleted_pre_trunc; /* tuples deleted prior to truncate */
- Oid t_id; /* table's OID */
- bool t_shared; /* is it a shared catalog? */
- bool t_truncated; /* was the relation truncated? */
-} TwoPhasePgStatRecord;
-
-/*
- * Info about current "snapshot" of stats file
- */
-static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
-
-/* Status for backends including auxiliary */
-static LocalPgBackendStatus *localBackendStatusTable = NULL;
-
-/* Total number of backends including auxiliary */
-static int localNumBackends = 0;
-
-/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
- */
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
-
-/* Signal handler flags */
-static volatile bool need_exit = false;
-static volatile bool got_SIGHUP = false;
-
-/*
- * Total time charged to functions so far in the current backend.
- * We use this to help separate "self" and "other" time charges.
- * (We assume this initializes to zero.)
- */
-static instr_time total_func_time;
-
-
-/* ----------
- * Local function forward declarations
- * ----------
- */
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-static void pgstat_exit(SIGNAL_ARGS);
-static void pgstat_beshutdown_hook(int code, Datum arg);
-static void pgstat_sighup_handler(SIGNAL_ARGS);
-
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
-static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
-
-static void pgstat_setup_memcxt(void);
-
-static const char *pgstat_get_wait_activity(WaitEventActivity w);
-static const char *pgstat_get_wait_client(WaitEventClient w);
-static const char *pgstat_get_wait_ipc(WaitEventIPC w);
-static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
-static const char *pgstat_get_wait_io(WaitEventIO w);
-
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
-/* ------------------------------------------------------------
- * Public functions called from postmaster follow
- * ------------------------------------------------------------
- */
-
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
- */
-void
-pgstat_init(void)
-{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
-
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
- {
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
- }
-
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
-
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
-
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
-
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
-
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
-
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- test_byte++; /* just make sure variable is changed */
-
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /* If we get here, we have a working socket */
- break;
- }
-
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
-
- /*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
- */
- if (!pg_set_noblock(pgStatSock))
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
- }
-
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
- }
-
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
-}
-
-/*
- * subroutine for pgstat_reset_all
- */
-static void
-pgstat_reset_remove_files(const char *directory)
-{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
-
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
- {
- int nchars;
- Oid tmp_oid;
-
- /*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
- */
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
-
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
- }
- FreeDir(dir);
-}
-
-/*
- * pgstat_reset_all() -
- *
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
- */
-void
-pgstat_reset_all(void)
-{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
-
-#ifdef EXEC_BACKEND
-
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
-
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
-
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
-}
-
-/* ------------------------------------------------------------
- * Public functions used by backends follow
- *------------------------------------------------------------
- */
-
-
-/* ----------
- * pgstat_report_stat() -
- *
- * Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
- * transaction stop time as an approximation of current time.
- * ----------
- */
-void
-pgstat_report_stat(bool force)
-{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
- TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
- int i;
-
- /* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
-
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
- now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
-
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
-
- /*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
- */
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
- {
- for (i = 0; i < tsa->tsa_used; i++)
- {
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
-
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
-
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
- continue;
-
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
- {
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
- }
- }
- /* zero out TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
- }
-
- /*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
- */
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
-
- /* Now, send function statistics */
- pgstat_send_funcstats();
-}
-
-/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
- */
-static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
-{
- int n;
- int len;
-
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
- */
- if (OidIsValid(tsmsg->m_databaseid))
- {
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
- pgStatXactCommit = 0;
- pgStatXactRollback = 0;
- pgStatBlockReadTime = 0;
- pgStatBlockWriteTime = 0;
- }
- else
- {
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
- }
-
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
-
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
-}
-
-/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
- */
-static void
-pgstat_send_funcstats(void)
-{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
- {
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
-
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
-}
-
-
-/* ----------
- * pgstat_vacuum_stat() -
- *
- * Will tell the collector about objects he can get rid of.
- * ----------
- */
-void
-pgstat_vacuum_stat(void)
-{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
- return;
-
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
-
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
- }
-}
-
-
-/* ----------
- * pgstat_collect_oids() -
- *
- * Collect the OIDs of all objects listed in the specified system catalog
- * into a temporary hash table. Caller should hash_destroy the result
- * when done with it. (However, we make the table in CurrentMemoryContext
- * so that it will be freed properly in event of an error.)
- * ----------
- */
-static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
-{
- HTAB *htab;
- HASHCTL hash_ctl;
- Relation rel;
- HeapScanDesc scan;
- HeapTuple tup;
- Snapshot snapshot;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(Oid);
- hash_ctl.hcxt = CurrentMemoryContext;
- htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- rel = table_open(catalogid, AccessShareLock);
- snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = heap_beginscan(rel, snapshot, 0, NULL);
- while ((tup = heap_getnext(scan, ForwardScanDirection)) != NULL)
- {
- Oid thisoid;
- bool isnull;
-
- thisoid = heap_getattr(tup, anum_oid, RelationGetDescr(rel), &isnull);
- Assert(!isnull);
-
- CHECK_FOR_INTERRUPTS();
-
- (void) hash_search(htab, (void *) &thisoid, HASH_ENTER, NULL);
- }
- heap_endscan(scan);
- UnregisterSnapshot(snapshot);
- table_close(rel, AccessShareLock);
-
- return htab;
-}
-
-
-/* ----------
- * pgstat_drop_database() -
- *
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
- */
-void
-pgstat_drop_database(Oid databaseid)
-{
- PgStat_MsgDropdb msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
-
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-}
-#endif /* NOT_USED */
-
-
-/* ----------
- * pgstat_reset_counters() -
- *
- * Tell the statistics collector to reset counters for our database.
- *
- * Permission checking for this function is managed through the normal
- * GRANT system.
- * ----------
- */
-void
-pgstat_reset_counters(void)
-{
- PgStat_MsgResetcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* ----------
- * pgstat_reset_shared_counters() -
- *
- * Tell the statistics collector to reset cluster-wide shared counters.
- *
- * Permission checking for this function is managed through the normal
- * GRANT system.
- * ----------
- */
-void
-pgstat_reset_shared_counters(const char *target)
-{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
- else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
- else
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("unrecognized reset target: \"%s\"", target),
- errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* ----------
- * pgstat_reset_single_counter() -
- *
- * Tell the statistics collector to reset a single counter.
- *
- * Permission checking for this function is managed through the normal
- * GRANT system.
- * ----------
- */
-void
-pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
-{
- PgStat_MsgResetsinglecounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
-
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* ----------
- * pgstat_report_autovac() -
- *
- * Called from autovacuum.c to report startup of an autovacuum process.
- * We are called before InitPostgres is done, so can't rely on MyDatabaseId;
- * the db OID must be passed in, instead.
- * ----------
- */
-void
-pgstat_report_autovac(Oid dboid)
-{
- PgStat_MsgAutovacStart msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
-
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ---------
- * pgstat_report_vacuum() -
- *
- * Tell the collector about the table we just vacuumed.
- * ---------
- */
-void
-pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
-{
- PgStat_MsgVacuum msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* --------
- * pgstat_report_analyze() -
- *
- * Tell the collector about the table we just analyzed.
- *
- * Caller must provide new live- and dead-tuples estimates, as well as a
- * flag indicating whether to reset the changes_since_analyze counter.
- * --------
- */
-void
-pgstat_report_analyze(Relation rel,
- PgStat_Counter livetuples, PgStat_Counter deadtuples,
- bool resetcounter)
-{
- PgStat_MsgAnalyze msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- /*
- * Unlike VACUUM, ANALYZE might be running inside a transaction that has
- * already inserted and/or deleted rows in the target table. ANALYZE will
- * have counted such rows as live or dead respectively. Because we will
- * report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
- */
- if (rel->pgstat_info != NULL)
- {
- PgStat_TableXactStatus *trans;
-
- for (trans = rel->pgstat_info->trans; trans; trans = trans->upper)
- {
- livetuples -= trans->tuples_inserted - trans->tuples_deleted;
- deadtuples -= trans->tuples_updated + trans->tuples_deleted;
- }
- /* count stuff inserted by already-aborted subxacts, too */
- deadtuples -= rel->pgstat_info->t_counts.t_delta_dead_tuples;
- /* Since ANALYZE's counts are estimates, we could have underflowed */
- livetuples = Max(livetuples, 0);
- deadtuples = Max(deadtuples, 0);
- }
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* --------
- * pgstat_report_recovery_conflict() -
- *
- * Tell the collector about a Hot Standby recovery conflict.
- * --------
- */
-void
-pgstat_report_recovery_conflict(int reason)
-{
- PgStat_MsgRecoveryConflict msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* --------
- * pgstat_report_deadlock() -
- *
- * Tell the collector about a deadlock detected.
- * --------
- */
-void
-pgstat_report_deadlock(void)
-{
- PgStat_MsgDeadlock msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* --------
- * pgstat_report_tempfile() -
- *
- * Tell the collector about a temporary file.
- * --------
- */
-void
-pgstat_report_tempfile(size_t filesize)
-{
- PgStat_MsgTempFile msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ----------
- * pgstat_ping() -
- *
- * Send some junk data to the collector to increase traffic.
- * ----------
- */
-void
-pgstat_ping(void)
-{
- PgStat_MsgDummy msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* ----------
- * pgstat_send_inquiry() -
- *
- * Notify collector that we need fresh data.
- * ----------
- */
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
-void
-pgstat_init_function_usage(FunctionCallInfo fcinfo,
- PgStat_FunctionCallUsage *fcu)
-{
- PgStat_BackendFunctionEntry *htabent;
- bool found;
-
- if (pgstat_track_functions <= fcinfo->flinfo->fn_stats)
- {
- /* stats not wanted */
- fcu->fs = NULL;
- return;
- }
-
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
- if (!found)
- MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
-
- fcu->fs = &htabent->f_counts;
-
- /* save stats for this function, later used to compensate for recursion */
- fcu->save_f_total_time = htabent->f_counts.f_total_time;
-
- /* save current backend-wide total time */
- fcu->save_total = total_func_time;
-
- /* get clock time as of function start */
- INSTR_TIME_SET_CURRENT(fcu->f_start);
-}
-
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
- *
- * If no entry, return NULL, don't create a new one
- */
-PgStat_BackendFunctionEntry *
-find_funcstat_entry(Oid func_id)
-{
- if (pgStatFunctions == NULL)
- return NULL;
-
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
-}
-
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
- *
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
- */
-void
-pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
-{
- PgStat_FunctionCounts *fs = fcu->fs;
- instr_time f_total;
- instr_time f_others;
- instr_time f_self;
-
- /* stats not wanted? */
- if (fs == NULL)
- return;
-
- /* total elapsed time in this function call */
- INSTR_TIME_SET_CURRENT(f_total);
- INSTR_TIME_SUBTRACT(f_total, fcu->f_start);
-
- /* self usage: elapsed minus anything already charged to other calls */
- f_others = total_func_time;
- INSTR_TIME_SUBTRACT(f_others, fcu->save_total);
- f_self = f_total;
- INSTR_TIME_SUBTRACT(f_self, f_others);
-
- /* update backend-wide total time */
- INSTR_TIME_ADD(total_func_time, f_self);
-
- /*
- * Compute the new f_total_time as the total elapsed time added to the
- * pre-call value of f_total_time. This is necessary to avoid
- * double-counting any time taken by recursive calls of myself. (We do
- * not need any similar kluge for self time, since that already excludes
- * any recursive calls.)
- */
- INSTR_TIME_ADD(f_total, fcu->save_f_total_time);
-
- /* update counters in function stats table */
- if (finalize)
- fs->f_numcalls++;
- fs->f_total_time = f_total;
- INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
-}
-
-
-/* ----------
- * pgstat_initstats() -
- *
- * Initialize a relcache entry to count access statistics.
- * Called whenever a relation is opened.
- *
- * We assume that a relcache entry's pgstat_info field is zeroed by
- * relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
- * ----------
- */
-void
-pgstat_initstats(Relation rel)
-{
- Oid rel_id = rel->rd_id;
- char relkind = rel->rd_rel->relkind;
-
- /* We only count stats for things that have storage */
- if (!(relkind == RELKIND_RELATION ||
- relkind == RELKIND_MATVIEW ||
- relkind == RELKIND_INDEX ||
- relkind == RELKIND_TOASTVALUE ||
- relkind == RELKIND_SEQUENCE))
- {
- rel->pgstat_info = NULL;
- return;
- }
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- {
- /* We're not counting at all */
- rel->pgstat_info = NULL;
- return;
- }
-
- /*
- * If we already set up this relation in the current transaction, nothing
- * to do.
- */
- if (rel->pgstat_info != NULL &&
- rel->pgstat_info->t_id == rel_id)
- return;
-
- /* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
-}
-
-/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
- */
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
-{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
-
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
- {
- HASHCTL ctl;
-
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /*
- * Find an entry or create a new one.
- */
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
- if (!found)
- {
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
- }
-
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
-
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
-
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
-
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
-
- return entry;
-}
-
-/*
- * find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
- *
- * If no entry, return NULL, don't create a new one
- *
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
- */
-PgStat_TableStatus *
-find_tabstat_entry(Oid rel_id)
-{
- TabStatHashEntry *hash_entry;
-
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
-
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
-}
-
-/*
- * get_tabstat_stack_level - add a new (sub)transaction stack entry if needed
- */
-static PgStat_SubXactStatus *
-get_tabstat_stack_level(int nest_level)
-{
- PgStat_SubXactStatus *xact_state;
-
- xact_state = pgStatXactStack;
- if (xact_state == NULL || xact_state->nest_level != nest_level)
- {
- xact_state = (PgStat_SubXactStatus *)
- MemoryContextAlloc(TopTransactionContext,
- sizeof(PgStat_SubXactStatus));
- xact_state->nest_level = nest_level;
- xact_state->prev = pgStatXactStack;
- xact_state->first = NULL;
- pgStatXactStack = xact_state;
- }
- return xact_state;
-}
-
-/*
- * add_tabstat_xact_level - add a new (sub)transaction state record
- */
-static void
-add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level)
-{
- PgStat_SubXactStatus *xact_state;
- PgStat_TableXactStatus *trans;
-
- /*
- * If this is the first rel to be modified at the current nest level, we
- * first have to push a transaction stack entry.
- */
- xact_state = get_tabstat_stack_level(nest_level);
-
- /* Now make a per-table stack entry */
- trans = (PgStat_TableXactStatus *)
- MemoryContextAllocZero(TopTransactionContext,
- sizeof(PgStat_TableXactStatus));
- trans->nest_level = nest_level;
- trans->upper = pgstat_info->trans;
- trans->parent = pgstat_info;
- trans->next = xact_state->first;
- xact_state->first = trans;
- pgstat_info->trans = trans;
-}
-
-/*
- * pgstat_count_heap_insert - count a tuple insertion of n tuples
- */
-void
-pgstat_count_heap_insert(Relation rel, PgStat_Counter n)
-{
- PgStat_TableStatus *pgstat_info = rel->pgstat_info;
-
- if (pgstat_info != NULL)
- {
- /* We have to log the effect at the proper transactional level */
- int nest_level = GetCurrentTransactionNestLevel();
-
- if (pgstat_info->trans == NULL ||
- pgstat_info->trans->nest_level != nest_level)
- add_tabstat_xact_level(pgstat_info, nest_level);
-
- pgstat_info->trans->tuples_inserted += n;
- }
-}
-
-/*
- * pgstat_count_heap_update - count a tuple update
- */
-void
-pgstat_count_heap_update(Relation rel, bool hot)
-{
- PgStat_TableStatus *pgstat_info = rel->pgstat_info;
-
- if (pgstat_info != NULL)
- {
- /* We have to log the effect at the proper transactional level */
- int nest_level = GetCurrentTransactionNestLevel();
-
- if (pgstat_info->trans == NULL ||
- pgstat_info->trans->nest_level != nest_level)
- add_tabstat_xact_level(pgstat_info, nest_level);
-
- pgstat_info->trans->tuples_updated++;
-
- /* t_tuples_hot_updated is nontransactional, so just advance it */
- if (hot)
- pgstat_info->t_counts.t_tuples_hot_updated++;
- }
-}
-
-/*
- * pgstat_count_heap_delete - count a tuple deletion
- */
-void
-pgstat_count_heap_delete(Relation rel)
-{
- PgStat_TableStatus *pgstat_info = rel->pgstat_info;
-
- if (pgstat_info != NULL)
- {
- /* We have to log the effect at the proper transactional level */
- int nest_level = GetCurrentTransactionNestLevel();
-
- if (pgstat_info->trans == NULL ||
- pgstat_info->trans->nest_level != nest_level)
- add_tabstat_xact_level(pgstat_info, nest_level);
-
- pgstat_info->trans->tuples_deleted++;
- }
-}
-
-/*
- * pgstat_truncate_save_counters
- *
- * Whenever a table is truncated, we save its i/u/d counters so that they can
- * be cleared, and if the (sub)xact that executed the truncate later aborts,
- * the counters can be restored to the saved (pre-truncate) values. Note we do
- * this on the first truncate in any particular subxact level only.
- */
-static void
-pgstat_truncate_save_counters(PgStat_TableXactStatus *trans)
-{
- if (!trans->truncated)
- {
- trans->inserted_pre_trunc = trans->tuples_inserted;
- trans->updated_pre_trunc = trans->tuples_updated;
- trans->deleted_pre_trunc = trans->tuples_deleted;
- trans->truncated = true;
- }
-}
-
-/*
- * pgstat_truncate_restore_counters - restore counters when a truncate aborts
- */
-static void
-pgstat_truncate_restore_counters(PgStat_TableXactStatus *trans)
-{
- if (trans->truncated)
- {
- trans->tuples_inserted = trans->inserted_pre_trunc;
- trans->tuples_updated = trans->updated_pre_trunc;
- trans->tuples_deleted = trans->deleted_pre_trunc;
- }
-}
-
-/*
- * pgstat_count_truncate - update tuple counters due to truncate
- */
-void
-pgstat_count_truncate(Relation rel)
-{
- PgStat_TableStatus *pgstat_info = rel->pgstat_info;
-
- if (pgstat_info != NULL)
- {
- /* We have to log the effect at the proper transactional level */
- int nest_level = GetCurrentTransactionNestLevel();
-
- if (pgstat_info->trans == NULL ||
- pgstat_info->trans->nest_level != nest_level)
- add_tabstat_xact_level(pgstat_info, nest_level);
-
- pgstat_truncate_save_counters(pgstat_info->trans);
- pgstat_info->trans->tuples_inserted = 0;
- pgstat_info->trans->tuples_updated = 0;
- pgstat_info->trans->tuples_deleted = 0;
- }
-}
-
-/*
- * pgstat_update_heap_dead_tuples - update dead-tuples count
- *
- * The semantics of this are that we are reporting the nontransactional
- * recovery of "delta" dead tuples; so t_delta_dead_tuples decreases
- * rather than increasing, and the change goes straight into the per-table
- * counter, not into transactional state.
- */
-void
-pgstat_update_heap_dead_tuples(Relation rel, int delta)
-{
- PgStat_TableStatus *pgstat_info = rel->pgstat_info;
-
- if (pgstat_info != NULL)
- pgstat_info->t_counts.t_delta_dead_tuples -= delta;
-}
-
-
-/* ----------
- * AtEOXact_PgStat
- *
- * Called from access/transam/xact.c at top-level transaction commit/abort.
- * ----------
- */
-void
-AtEOXact_PgStat(bool isCommit)
-{
- PgStat_SubXactStatus *xact_state;
-
- /*
- * Count transaction commit or abort. (We use counters, not just bools,
- * in case the reporting message isn't sent right away.)
- */
- if (isCommit)
- pgStatXactCommit++;
- else
- pgStatXactRollback++;
-
- /*
- * Transfer transactional insert/update counts into the base tabstat
- * entries. We don't bother to free any of the transactional state, since
- * it's all in TopTransactionContext and will go away anyway.
- */
- xact_state = pgStatXactStack;
- if (xact_state != NULL)
- {
- PgStat_TableXactStatus *trans;
-
- Assert(xact_state->nest_level == 1);
- Assert(xact_state->prev == NULL);
- for (trans = xact_state->first; trans != NULL; trans = trans->next)
- {
- PgStat_TableStatus *tabstat;
-
- Assert(trans->nest_level == 1);
- Assert(trans->upper == NULL);
- tabstat = trans->parent;
- Assert(tabstat->trans == trans);
- /* restore pre-truncate stats (if any) in case of aborted xact */
- if (!isCommit)
- pgstat_truncate_restore_counters(trans);
- /* count attempted actions regardless of commit/abort */
- tabstat->t_counts.t_tuples_inserted += trans->tuples_inserted;
- tabstat->t_counts.t_tuples_updated += trans->tuples_updated;
- tabstat->t_counts.t_tuples_deleted += trans->tuples_deleted;
- if (isCommit)
- {
- tabstat->t_counts.t_truncated = trans->truncated;
- if (trans->truncated)
- {
- /* forget live/dead stats seen by backend thus far */
- tabstat->t_counts.t_delta_live_tuples = 0;
- tabstat->t_counts.t_delta_dead_tuples = 0;
- }
- /* insert adds a live tuple, delete removes one */
- tabstat->t_counts.t_delta_live_tuples +=
- trans->tuples_inserted - trans->tuples_deleted;
- /* update and delete each create a dead tuple */
- tabstat->t_counts.t_delta_dead_tuples +=
- trans->tuples_updated + trans->tuples_deleted;
- /* insert, update, delete each count as one change event */
- tabstat->t_counts.t_changed_tuples +=
- trans->tuples_inserted + trans->tuples_updated +
- trans->tuples_deleted;
- }
- else
- {
- /* inserted tuples are dead, deleted tuples are unaffected */
- tabstat->t_counts.t_delta_dead_tuples +=
- trans->tuples_inserted + trans->tuples_updated;
- /* an aborted xact generates no changed_tuple events */
- }
- tabstat->trans = NULL;
- }
- }
- pgStatXactStack = NULL;
-
- /* Make sure any stats snapshot is thrown away */
- pgstat_clear_snapshot();
-}
-
-/* ----------
- * AtEOSubXact_PgStat
- *
- * Called from access/transam/xact.c at subtransaction commit/abort.
- * ----------
- */
-void
-AtEOSubXact_PgStat(bool isCommit, int nestDepth)
-{
- PgStat_SubXactStatus *xact_state;
-
- /*
- * Transfer transactional insert/update counts into the next higher
- * subtransaction state.
- */
- xact_state = pgStatXactStack;
- if (xact_state != NULL &&
- xact_state->nest_level >= nestDepth)
- {
- PgStat_TableXactStatus *trans;
- PgStat_TableXactStatus *next_trans;
-
- /* delink xact_state from stack immediately to simplify reuse case */
- pgStatXactStack = xact_state->prev;
-
- for (trans = xact_state->first; trans != NULL; trans = next_trans)
- {
- PgStat_TableStatus *tabstat;
-
- next_trans = trans->next;
- Assert(trans->nest_level == nestDepth);
- tabstat = trans->parent;
- Assert(tabstat->trans == trans);
- if (isCommit)
- {
- if (trans->upper && trans->upper->nest_level == nestDepth - 1)
- {
- if (trans->truncated)
- {
- /* propagate the truncate status one level up */
- pgstat_truncate_save_counters(trans->upper);
- /* replace upper xact stats with ours */
- trans->upper->tuples_inserted = trans->tuples_inserted;
- trans->upper->tuples_updated = trans->tuples_updated;
- trans->upper->tuples_deleted = trans->tuples_deleted;
- }
- else
- {
- trans->upper->tuples_inserted += trans->tuples_inserted;
- trans->upper->tuples_updated += trans->tuples_updated;
- trans->upper->tuples_deleted += trans->tuples_deleted;
- }
- tabstat->trans = trans->upper;
- pfree(trans);
- }
- else
- {
- /*
- * When there isn't an immediate parent state, we can just
- * reuse the record instead of going through a
- * palloc/pfree pushup (this works since it's all in
- * TopTransactionContext anyway). We have to re-link it
- * into the parent level, though, and that might mean
- * pushing a new entry into the pgStatXactStack.
- */
- PgStat_SubXactStatus *upper_xact_state;
-
- upper_xact_state = get_tabstat_stack_level(nestDepth - 1);
- trans->next = upper_xact_state->first;
- upper_xact_state->first = trans;
- trans->nest_level = nestDepth - 1;
- }
- }
- else
- {
- /*
- * On abort, update top-level tabstat counts, then forget the
- * subtransaction
- */
-
- /* first restore values obliterated by truncate */
- pgstat_truncate_restore_counters(trans);
- /* count attempted actions regardless of commit/abort */
- tabstat->t_counts.t_tuples_inserted += trans->tuples_inserted;
- tabstat->t_counts.t_tuples_updated += trans->tuples_updated;
- tabstat->t_counts.t_tuples_deleted += trans->tuples_deleted;
- /* inserted tuples are dead, deleted tuples are unaffected */
- tabstat->t_counts.t_delta_dead_tuples +=
- trans->tuples_inserted + trans->tuples_updated;
- tabstat->trans = trans->upper;
- pfree(trans);
- }
- }
- pfree(xact_state);
- }
-}
-
-
-/*
- * AtPrepare_PgStat
- * Save the transactional stats state at 2PC transaction prepare.
- *
- * In this phase we just generate 2PC records for all the pending
- * transaction-dependent stats work.
- */
-void
-AtPrepare_PgStat(void)
-{
- PgStat_SubXactStatus *xact_state;
-
- xact_state = pgStatXactStack;
- if (xact_state != NULL)
- {
- PgStat_TableXactStatus *trans;
-
- Assert(xact_state->nest_level == 1);
- Assert(xact_state->prev == NULL);
- for (trans = xact_state->first; trans != NULL; trans = trans->next)
- {
- PgStat_TableStatus *tabstat;
- TwoPhasePgStatRecord record;
-
- Assert(trans->nest_level == 1);
- Assert(trans->upper == NULL);
- tabstat = trans->parent;
- Assert(tabstat->trans == trans);
-
- record.tuples_inserted = trans->tuples_inserted;
- record.tuples_updated = trans->tuples_updated;
- record.tuples_deleted = trans->tuples_deleted;
- record.inserted_pre_trunc = trans->inserted_pre_trunc;
- record.updated_pre_trunc = trans->updated_pre_trunc;
- record.deleted_pre_trunc = trans->deleted_pre_trunc;
- record.t_id = tabstat->t_id;
- record.t_shared = tabstat->t_shared;
- record.t_truncated = trans->truncated;
-
- RegisterTwoPhaseRecord(TWOPHASE_RM_PGSTAT_ID, 0,
- &record, sizeof(TwoPhasePgStatRecord));
- }
- }
-}
-
-/*
- * PostPrepare_PgStat
- * Clean up after successful PREPARE.
- *
- * All we need do here is unlink the transaction stats state from the
- * nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
- *
- * Note: AtEOXact_PgStat is not called during PREPARE.
- */
-void
-PostPrepare_PgStat(void)
-{
- PgStat_SubXactStatus *xact_state;
-
- /*
- * We don't bother to free any of the transactional state, since it's all
- * in TopTransactionContext and will go away anyway.
- */
- xact_state = pgStatXactStack;
- if (xact_state != NULL)
- {
- PgStat_TableXactStatus *trans;
-
- for (trans = xact_state->first; trans != NULL; trans = trans->next)
- {
- PgStat_TableStatus *tabstat;
-
- tabstat = trans->parent;
- tabstat->trans = NULL;
- }
- }
- pgStatXactStack = NULL;
-
- /* Make sure any stats snapshot is thrown away */
- pgstat_clear_snapshot();
-}
-
-/*
- * 2PC processing routine for COMMIT PREPARED case.
- *
- * Load the saved counts into our local pgstats state.
- */
-void
-pgstat_twophase_postcommit(TransactionId xid, uint16 info,
- void *recdata, uint32 len)
-{
- TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata;
- PgStat_TableStatus *pgstat_info;
-
- /* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
-
- /* Same math as in AtEOXact_PgStat, commit case */
- pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
- pgstat_info->t_counts.t_tuples_updated += rec->tuples_updated;
- pgstat_info->t_counts.t_tuples_deleted += rec->tuples_deleted;
- pgstat_info->t_counts.t_truncated = rec->t_truncated;
- if (rec->t_truncated)
- {
- /* forget live/dead stats seen by backend thus far */
- pgstat_info->t_counts.t_delta_live_tuples = 0;
- pgstat_info->t_counts.t_delta_dead_tuples = 0;
- }
- pgstat_info->t_counts.t_delta_live_tuples +=
- rec->tuples_inserted - rec->tuples_deleted;
- pgstat_info->t_counts.t_delta_dead_tuples +=
- rec->tuples_updated + rec->tuples_deleted;
- pgstat_info->t_counts.t_changed_tuples +=
- rec->tuples_inserted + rec->tuples_updated +
- rec->tuples_deleted;
-}
-
-/*
- * 2PC processing routine for ROLLBACK PREPARED case.
- *
- * Load the saved counts into our local pgstats state, but treat them
- * as aborted.
- */
-void
-pgstat_twophase_postabort(TransactionId xid, uint16 info,
- void *recdata, uint32 len)
-{
- TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata;
- PgStat_TableStatus *pgstat_info;
-
- /* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
-
- /* Same math as in AtEOXact_PgStat, abort case */
- if (rec->t_truncated)
- {
- rec->tuples_inserted = rec->inserted_pre_trunc;
- rec->tuples_updated = rec->updated_pre_trunc;
- rec->tuples_deleted = rec->deleted_pre_trunc;
- }
- pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
- pgstat_info->t_counts.t_tuples_updated += rec->tuples_updated;
- pgstat_info->t_counts.t_tuples_deleted += rec->tuples_deleted;
- pgstat_info->t_counts.t_delta_dead_tuples +=
- rec->tuples_inserted + rec->tuples_updated;
-}
-
-
-/* ----------
- * pgstat_fetch_stat_dbentry() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
- * ----------
- */
-PgStat_StatDBEntry *
-pgstat_fetch_stat_dbentry(Oid dbid)
-{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
-}
-
-
-/* ----------
- * pgstat_fetch_stat_tabentry() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
- * that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
- * ----------
- */
-PgStat_StatTabEntry *
-pgstat_fetch_stat_tabentry(Oid relid)
-{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- /*
- * If we didn't find it, maybe it's a shared table.
- */
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
-}
-
-
-/* ----------
- * pgstat_fetch_stat_funcentry() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one function or NULL.
- * ----------
- */
-PgStat_StatFuncEntry *
-pgstat_fetch_stat_funcentry(Oid func_id)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
-
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
-
- return funcentry;
-}
-
-
-/* ----------
- * pgstat_fetch_stat_beentry() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * our local copy of the current-activity entry for one backend.
- *
- * NB: caller is responsible for a check if the user is permitted to see
- * this info (especially the querystring).
- * ----------
- */
-PgBackendStatus *
-pgstat_fetch_stat_beentry(int beid)
-{
- pgstat_read_current_status();
-
- if (beid < 1 || beid > localNumBackends)
- return NULL;
-
- return &localBackendStatusTable[beid - 1].backendStatus;
-}
-
-
-/* ----------
- * pgstat_fetch_stat_local_beentry() -
- *
- * Like pgstat_fetch_stat_beentry() but with locally computed additions (like
- * xid and xmin values of the backend)
- *
- * NB: caller is responsible for a check if the user is permitted to see
- * this info (especially the querystring).
- * ----------
- */
-LocalPgBackendStatus *
-pgstat_fetch_stat_local_beentry(int beid)
-{
- pgstat_read_current_status();
-
- if (beid < 1 || beid > localNumBackends)
- return NULL;
-
- return &localBackendStatusTable[beid - 1];
-}
-
-
-/* ----------
- * pgstat_fetch_stat_numbackends() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * the maximum current backend id.
- * ----------
- */
-int
-pgstat_fetch_stat_numbackends(void)
-{
- pgstat_read_current_status();
-
- return localNumBackends;
-}
-
-/*
- * ---------
- * pgstat_fetch_stat_archiver() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the archiver statistics struct.
- * ---------
- */
-PgStat_ArchiverStats *
-pgstat_fetch_stat_archiver(void)
-{
- backend_read_statsfile();
-
- return &archiverStats;
-}
-
-
-/*
- * ---------
- * pgstat_fetch_global() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the global statistics struct.
- * ---------
- */
-PgStat_GlobalStats *
-pgstat_fetch_global(void)
-{
- backend_read_statsfile();
-
- return &globalStats;
-}
-
-
-/* ------------------------------------------------------------
- * Functions for management of the shared-memory PgBackendStatus array
- * ------------------------------------------------------------
- */
-
-static PgBackendStatus *BackendStatusArray = NULL;
-static PgBackendStatus *MyBEEntry = NULL;
-static char *BackendAppnameBuffer = NULL;
-static char *BackendClientHostnameBuffer = NULL;
-static char *BackendActivityBuffer = NULL;
-static Size BackendActivityBufferSize = 0;
-#ifdef USE_SSL
-static PgBackendSSLStatus *BackendSslStatusBuffer = NULL;
-#endif
-
-
-/*
- * Report shared-memory space needed by CreateSharedBackendStatus.
- */
-Size
-BackendStatusShmemSize(void)
-{
- Size size;
-
- /* BackendStatusArray: */
- size = mul_size(sizeof(PgBackendStatus), NumBackendStatSlots);
- /* BackendAppnameBuffer: */
- size = add_size(size,
- mul_size(NAMEDATALEN, NumBackendStatSlots));
- /* BackendClientHostnameBuffer: */
- size = add_size(size,
- mul_size(NAMEDATALEN, NumBackendStatSlots));
- /* BackendActivityBuffer: */
- size = add_size(size,
- mul_size(pgstat_track_activity_query_size, NumBackendStatSlots));
-#ifdef USE_SSL
- /* BackendSslStatusBuffer: */
- size = add_size(size,
- mul_size(sizeof(PgBackendSSLStatus), NumBackendStatSlots));
-#endif
- return size;
-}
-
-/*
- * Initialize the shared status array and several string buffers
- * during postmaster startup.
- */
-void
-CreateSharedBackendStatus(void)
-{
- Size size;
- bool found;
- int i;
- char *buffer;
-
- /* Create or attach to the shared array */
- size = mul_size(sizeof(PgBackendStatus), NumBackendStatSlots);
- BackendStatusArray = (PgBackendStatus *)
- ShmemInitStruct("Backend Status Array", size, &found);
-
- if (!found)
- {
- /*
- * We're the first - initialize.
- */
- MemSet(BackendStatusArray, 0, size);
- }
-
- /* Create or attach to the shared appname buffer */
- size = mul_size(NAMEDATALEN, NumBackendStatSlots);
- BackendAppnameBuffer = (char *)
- ShmemInitStruct("Backend Application Name Buffer", size, &found);
-
- if (!found)
- {
- MemSet(BackendAppnameBuffer, 0, size);
-
- /* Initialize st_appname pointers. */
- buffer = BackendAppnameBuffer;
- for (i = 0; i < NumBackendStatSlots; i++)
- {
- BackendStatusArray[i].st_appname = buffer;
- buffer += NAMEDATALEN;
- }
- }
-
- /* Create or attach to the shared client hostname buffer */
- size = mul_size(NAMEDATALEN, NumBackendStatSlots);
- BackendClientHostnameBuffer = (char *)
- ShmemInitStruct("Backend Client Host Name Buffer", size, &found);
-
- if (!found)
- {
- MemSet(BackendClientHostnameBuffer, 0, size);
-
- /* Initialize st_clienthostname pointers. */
- buffer = BackendClientHostnameBuffer;
- for (i = 0; i < NumBackendStatSlots; i++)
- {
- BackendStatusArray[i].st_clienthostname = buffer;
- buffer += NAMEDATALEN;
- }
- }
-
- /* Create or attach to the shared activity buffer */
- BackendActivityBufferSize = mul_size(pgstat_track_activity_query_size,
- NumBackendStatSlots);
- BackendActivityBuffer = (char *)
- ShmemInitStruct("Backend Activity Buffer",
- BackendActivityBufferSize,
- &found);
-
- if (!found)
- {
- MemSet(BackendActivityBuffer, 0, BackendActivityBufferSize);
-
- /* Initialize st_activity pointers. */
- buffer = BackendActivityBuffer;
- for (i = 0; i < NumBackendStatSlots; i++)
- {
- BackendStatusArray[i].st_activity_raw = buffer;
- buffer += pgstat_track_activity_query_size;
- }
- }
-
-#ifdef USE_SSL
- /* Create or attach to the shared SSL status buffer */
- size = mul_size(sizeof(PgBackendSSLStatus), NumBackendStatSlots);
- BackendSslStatusBuffer = (PgBackendSSLStatus *)
- ShmemInitStruct("Backend SSL Status Buffer", size, &found);
-
- if (!found)
- {
- PgBackendSSLStatus *ptr;
-
- MemSet(BackendSslStatusBuffer, 0, size);
-
- /* Initialize st_sslstatus pointers. */
- ptr = BackendSslStatusBuffer;
- for (i = 0; i < NumBackendStatSlots; i++)
- {
- BackendStatusArray[i].st_sslstatus = ptr;
- ptr++;
- }
- }
-#endif
-}
-
-
-/* ----------
- * pgstat_initialize() -
- *
- * Initialize pgstats state, and set up our on-proc-exit hook.
- * Called from InitPostgres and AuxiliaryProcessMain. For auxiliary process,
- * MyBackendId is invalid. Otherwise, MyBackendId must be set,
- * but we must not have started any transaction yet (since the
- * exit hook must run after the last transaction exit).
- * NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
- * ----------
- */
-void
-pgstat_initialize(void)
-{
- /* Initialize MyBEEntry */
- if (MyBackendId != InvalidBackendId)
- {
- Assert(MyBackendId >= 1 && MyBackendId <= MaxBackends);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
-
- /*
- * Assign the MyBEEntry for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 1 to MaxBackends (inclusive), so we use
- * MaxBackends + AuxBackendType + 1 as the index of the slot for an
- * auxiliary process.
- */
- MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
- }
-
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
-}
-
-/* ----------
- * pgstat_bestart() -
- *
- * Initialize this backend's entry in the PgBackendStatus array.
- * Called from InitPostgres.
- *
- * Apart from auxiliary processes, MyBackendId, MyDatabaseId,
- * session userid, and application_name must be set for a
- * backend (hence, this cannot be combined with pgstat_initialize).
- * ----------
- */
-void
-pgstat_bestart(void)
-{
- SockAddr clientaddr;
- volatile PgBackendStatus *beentry;
-
- /*
- * To minimize the time spent modifying the PgBackendStatus entry, fetch
- * all the needed data first.
- */
-
- /*
- * We may not have a MyProcPort (eg, if this is the autovacuum process).
- * If so, use all-zeroes client address, which is dealt with specially in
- * pg_stat_get_backend_client_addr and pg_stat_get_backend_client_port.
- */
- if (MyProcPort)
- memcpy(&clientaddr, &MyProcPort->raddr, sizeof(clientaddr));
- else
- MemSet(&clientaddr, 0, sizeof(clientaddr));
-
- /*
- * Initialize my status entry, following the protocol of bumping
- * st_changecount before and after; and make sure it's even afterwards. We
- * use a volatile pointer here to ensure the compiler doesn't try to get
- * cute.
- */
- beentry = MyBEEntry;
-
- /* pgstats state must be initialized from pgstat_initialize() */
- Assert(beentry != NULL);
-
- if (MyBackendId != InvalidBackendId)
- {
- if (IsAutoVacuumLauncherProcess())
- {
- /* Autovacuum Launcher */
- beentry->st_backendType = B_AUTOVAC_LAUNCHER;
- }
- else if (IsAutoVacuumWorkerProcess())
- {
- /* Autovacuum Worker */
- beentry->st_backendType = B_AUTOVAC_WORKER;
- }
- else if (am_walsender)
- {
- /* Wal sender */
- beentry->st_backendType = B_WAL_SENDER;
- }
- else if (IsBackgroundWorker)
- {
- /* bgworker */
- beentry->st_backendType = B_BG_WORKER;
- }
- else
- {
- /* client-backend */
- beentry->st_backendType = B_BACKEND;
- }
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
- switch (MyAuxProcType)
- {
- case StartupProcess:
- beentry->st_backendType = B_STARTUP;
- break;
- case BgWriterProcess:
- beentry->st_backendType = B_BG_WRITER;
- break;
- case ArchiverProcess:
- beentry->st_backendType = B_ARCHIVER;
- break;
- case CheckpointerProcess:
- beentry->st_backendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- beentry->st_backendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- beentry->st_backendType = B_WAL_RECEIVER;
- break;
- default:
- elog(FATAL, "unrecognized process type: %d",
- (int) MyAuxProcType);
- proc_exit(1);
- }
- }
-
- do
- {
- pgstat_increment_changecount_before(beentry);
- } while ((beentry->st_changecount & 1) == 0);
-
- beentry->st_procpid = MyProcPid;
- beentry->st_proc_start_timestamp = MyStartTimestamp;
- beentry->st_activity_start_timestamp = 0;
- beentry->st_state_start_timestamp = 0;
- beentry->st_xact_start_timestamp = 0;
- beentry->st_databaseid = MyDatabaseId;
-
- /* We have userid for client-backends, wal-sender and bgworker processes */
- if (beentry->st_backendType == B_BACKEND
- || beentry->st_backendType == B_WAL_SENDER
- || beentry->st_backendType == B_BG_WORKER)
- beentry->st_userid = GetSessionUserId();
- else
- beentry->st_userid = InvalidOid;
-
- beentry->st_clientaddr = clientaddr;
- if (MyProcPort && MyProcPort->remote_hostname)
- strlcpy(beentry->st_clienthostname, MyProcPort->remote_hostname,
- NAMEDATALEN);
- else
- beentry->st_clienthostname[0] = '\0';
-#ifdef USE_SSL
- if (MyProcPort && MyProcPort->ssl != NULL)
- {
- beentry->st_ssl = true;
- beentry->st_sslstatus->ssl_bits = be_tls_get_cipher_bits(MyProcPort);
- beentry->st_sslstatus->ssl_compression = be_tls_get_compression(MyProcPort);
- strlcpy(beentry->st_sslstatus->ssl_version, be_tls_get_version(MyProcPort), NAMEDATALEN);
- strlcpy(beentry->st_sslstatus->ssl_cipher, be_tls_get_cipher(MyProcPort), NAMEDATALEN);
- be_tls_get_peer_subject_name(MyProcPort, beentry->st_sslstatus->ssl_client_dn, NAMEDATALEN);
- be_tls_get_peer_serial(MyProcPort, beentry->st_sslstatus->ssl_client_serial, NAMEDATALEN);
- be_tls_get_peer_issuer_name(MyProcPort, beentry->st_sslstatus->ssl_issuer_dn, NAMEDATALEN);
- }
- else
- {
- beentry->st_ssl = false;
- }
-#else
- beentry->st_ssl = false;
-#endif
- beentry->st_state = STATE_UNDEFINED;
- beentry->st_appname[0] = '\0';
- beentry->st_activity_raw[0] = '\0';
- /* Also make sure the last byte in each string area is always 0 */
- beentry->st_clienthostname[NAMEDATALEN - 1] = '\0';
- beentry->st_appname[NAMEDATALEN - 1] = '\0';
- beentry->st_activity_raw[pgstat_track_activity_query_size - 1] = '\0';
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
-
- /*
- * we don't zero st_progress_param here to save cycles; nobody should
- * examine it until st_progress_command has been set to something other
- * than PROGRESS_COMMAND_INVALID
- */
-
- pgstat_increment_changecount_after(beentry);
-
- /* Update app name to current GUC setting */
- if (application_name)
- pgstat_report_appname(application_name);
-}
-
-/*
- * Shut down a single backend's statistics reporting at process exit.
- *
- * Flush any remaining statistics counts out to the collector.
- * Without this, operations triggered during backend exit (such as
- * temp table deletions) won't be counted.
- *
- * Lastly, clear out our entry in the PgBackendStatus array.
- */
-static void
-pgstat_beshutdown_hook(int code, Datum arg)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
-
- /*
- * If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
- * database ID, so forget it. (This means that accesses to pg_database
- * during failed backend starts might never get counted.)
- */
- if (OidIsValid(MyDatabaseId))
- pgstat_report_stat(true);
-
- /*
- * Clear my status entry, following the protocol of bumping st_changecount
- * before and after. We use a volatile pointer here to ensure the
- * compiler doesn't try to get cute.
- */
- pgstat_increment_changecount_before(beentry);
-
- beentry->st_procpid = 0; /* mark invalid */
-
- pgstat_increment_changecount_after(beentry);
-}
-
-
-/* ----------
- * pgstat_report_activity() -
- *
- * Called from tcop/postgres.c to report what the backend is actually doing
- * (but note cmd_str can be NULL for certain cases).
- *
- * All updates of the status entry follow the protocol of bumping
- * st_changecount before and after. We use a volatile pointer here to
- * ensure the compiler doesn't try to get cute.
- * ----------
- */
-void
-pgstat_report_activity(BackendState state, const char *cmd_str)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
- TimestampTz start_timestamp;
- TimestampTz current_timestamp;
- int len = 0;
-
- TRACE_POSTGRESQL_STATEMENT_STATUS(cmd_str);
-
- if (!beentry)
- return;
-
- if (!pgstat_track_activities)
- {
- if (beentry->st_state != STATE_DISABLED)
- {
- volatile PGPROC *proc = MyProc;
-
- /*
- * track_activities is disabled, but we last reported a
- * non-disabled state. As our final update, change the state and
- * clear fields we will not be updating anymore.
- */
- pgstat_increment_changecount_before(beentry);
- beentry->st_state = STATE_DISABLED;
- beentry->st_state_start_timestamp = 0;
- beentry->st_activity_raw[0] = '\0';
- beentry->st_activity_start_timestamp = 0;
- /* st_xact_start_timestamp and wait_event_info are also disabled */
- beentry->st_xact_start_timestamp = 0;
- proc->wait_event_info = 0;
- pgstat_increment_changecount_after(beentry);
- }
- return;
- }
-
- /*
- * To minimize the time spent modifying the entry, fetch all the needed
- * data first.
- */
- start_timestamp = GetCurrentStatementStartTimestamp();
- if (cmd_str != NULL)
- {
- /*
- * Compute length of to-be-stored string unaware of multi-byte
- * characters. For speed reasons that'll get corrected on read, rather
- * than computed every write.
- */
- len = Min(strlen(cmd_str), pgstat_track_activity_query_size - 1);
- }
- current_timestamp = GetCurrentTimestamp();
-
- /*
- * Now update the status entry
- */
- pgstat_increment_changecount_before(beentry);
-
- beentry->st_state = state;
- beentry->st_state_start_timestamp = current_timestamp;
-
- if (cmd_str != NULL)
- {
- memcpy((char *) beentry->st_activity_raw, cmd_str, len);
- beentry->st_activity_raw[len] = '\0';
- beentry->st_activity_start_timestamp = start_timestamp;
- }
-
- pgstat_increment_changecount_after(beentry);
-}
-
-/*-----------
- * pgstat_progress_start_command() -
- *
- * Set st_progress_command (and st_progress_command_target) in own backend
- * entry. Also, zero-initialize st_progress_param array.
- *-----------
- */
-void
-pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
-
- if (!beentry || !pgstat_track_activities)
- return;
-
- pgstat_increment_changecount_before(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
- pgstat_increment_changecount_after(beentry);
-}
-
-/*-----------
- * pgstat_progress_update_param() -
- *
- * Update index'th member in st_progress_param[] of own backend entry.
- *-----------
- */
-void
-pgstat_progress_update_param(int index, int64 val)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
-
- Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
-
- if (!beentry || !pgstat_track_activities)
- return;
-
- pgstat_increment_changecount_before(beentry);
- beentry->st_progress_param[index] = val;
- pgstat_increment_changecount_after(beentry);
-}
-
-/*-----------
- * pgstat_progress_update_multi_param() -
- *
- * Update multiple members in st_progress_param[] of own backend entry.
- * This is atomic; readers won't see intermediate states.
- *-----------
- */
-void
-pgstat_progress_update_multi_param(int nparam, const int *index,
- const int64 *val)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
- int i;
-
- if (!beentry || !pgstat_track_activities || nparam == 0)
- return;
-
- pgstat_increment_changecount_before(beentry);
-
- for (i = 0; i < nparam; ++i)
- {
- Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
-
- beentry->st_progress_param[index[i]] = val[i];
- }
-
- pgstat_increment_changecount_after(beentry);
-}
-
-/*-----------
- * pgstat_progress_end_command() -
- *
- * Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
- *-----------
- */
-void
-pgstat_progress_end_command(void)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
-
- if (!beentry)
- return;
- if (!pgstat_track_activities
- && beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
- pgstat_increment_changecount_before(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
- pgstat_increment_changecount_after(beentry);
-}
-
-/* ----------
- * pgstat_report_appname() -
- *
- * Called to update our application name.
- * ----------
- */
-void
-pgstat_report_appname(const char *appname)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
- int len;
-
- if (!beentry)
- return;
-
- /* This should be unnecessary if GUC did its job, but be safe */
- len = pg_mbcliplen(appname, strlen(appname), NAMEDATALEN - 1);
-
- /*
- * Update my status entry, following the protocol of bumping
- * st_changecount before and after. We use a volatile pointer here to
- * ensure the compiler doesn't try to get cute.
- */
- pgstat_increment_changecount_before(beentry);
-
- memcpy((char *) beentry->st_appname, appname, len);
- beentry->st_appname[len] = '\0';
-
- pgstat_increment_changecount_after(beentry);
-}
-
-/*
- * Report current transaction start timestamp as the specified value.
- * Zero means there is no active transaction.
- */
-void
-pgstat_report_xact_timestamp(TimestampTz tstamp)
-{
- volatile PgBackendStatus *beentry = MyBEEntry;
-
- if (!pgstat_track_activities || !beentry)
- return;
-
- /*
- * Update my status entry, following the protocol of bumping
- * st_changecount before and after. We use a volatile pointer here to
- * ensure the compiler doesn't try to get cute.
- */
- pgstat_increment_changecount_before(beentry);
- beentry->st_xact_start_timestamp = tstamp;
- pgstat_increment_changecount_after(beentry);
-}
-
-/* ----------
- * pgstat_read_current_status() -
- *
- * Copy the current contents of the PgBackendStatus array to local memory,
- * if not already done in this transaction.
- * ----------
- */
-static void
-pgstat_read_current_status(void)
-{
- volatile PgBackendStatus *beentry;
- LocalPgBackendStatus *localtable;
- LocalPgBackendStatus *localentry;
- char *localappname,
- *localclienthostname,
- *localactivity;
-#ifdef USE_SSL
- PgBackendSSLStatus *localsslstatus;
-#endif
- int i;
-
- Assert(!pgStatRunningInCollector);
- if (localBackendStatusTable)
- return; /* already done */
-
- pgstat_setup_memcxt();
-
- localtable = (LocalPgBackendStatus *)
- MemoryContextAlloc(pgStatLocalContext,
- sizeof(LocalPgBackendStatus) * NumBackendStatSlots);
- localappname = (char *)
- MemoryContextAlloc(pgStatLocalContext,
- NAMEDATALEN * NumBackendStatSlots);
- localclienthostname = (char *)
- MemoryContextAlloc(pgStatLocalContext,
- NAMEDATALEN * NumBackendStatSlots);
- localactivity = (char *)
- MemoryContextAlloc(pgStatLocalContext,
- pgstat_track_activity_query_size * NumBackendStatSlots);
-#ifdef USE_SSL
- localsslstatus = (PgBackendSSLStatus *)
- MemoryContextAlloc(pgStatLocalContext,
- sizeof(PgBackendSSLStatus) * NumBackendStatSlots);
-#endif
-
- localNumBackends = 0;
-
- beentry = BackendStatusArray;
- localentry = localtable;
- for (i = 1; i <= NumBackendStatSlots; i++)
- {
- /*
- * Follow the protocol of retrying if st_changecount changes while we
- * copy the entry, or if it's odd. (The check for odd is needed to
- * cover the case where we are able to completely copy the entry while
- * the source backend is between increment steps.) We use a volatile
- * pointer here to ensure the compiler doesn't try to get cute.
- */
- for (;;)
- {
- int before_changecount;
- int after_changecount;
-
- pgstat_save_changecount_before(beentry, before_changecount);
-
- localentry->backendStatus.st_procpid = beentry->st_procpid;
- if (localentry->backendStatus.st_procpid > 0)
- {
- memcpy(&localentry->backendStatus, (char *) beentry, sizeof(PgBackendStatus));
-
- /*
- * strcpy is safe even if the string is modified concurrently,
- * because there's always a \0 at the end of the buffer.
- */
- strcpy(localappname, (char *) beentry->st_appname);
- localentry->backendStatus.st_appname = localappname;
- strcpy(localclienthostname, (char *) beentry->st_clienthostname);
- localentry->backendStatus.st_clienthostname = localclienthostname;
- strcpy(localactivity, (char *) beentry->st_activity_raw);
- localentry->backendStatus.st_activity_raw = localactivity;
- localentry->backendStatus.st_ssl = beentry->st_ssl;
-#ifdef USE_SSL
- if (beentry->st_ssl)
- {
- memcpy(localsslstatus, beentry->st_sslstatus, sizeof(PgBackendSSLStatus));
- localentry->backendStatus.st_sslstatus = localsslstatus;
- }
-#endif
- }
-
- pgstat_save_changecount_after(beentry, after_changecount);
- if (before_changecount == after_changecount &&
- (before_changecount & 1) == 0)
- break;
-
- /* Make sure we can break out of loop if stuck... */
- CHECK_FOR_INTERRUPTS();
- }
-
- beentry++;
- /* Only valid entries get included into the local array */
- if (localentry->backendStatus.st_procpid > 0)
- {
- BackendIdGetTransactionIds(i,
- &localentry->backend_xid,
- &localentry->backend_xmin);
-
- localentry++;
- localappname += NAMEDATALEN;
- localclienthostname += NAMEDATALEN;
- localactivity += pgstat_track_activity_query_size;
-#ifdef USE_SSL
- localsslstatus++;
-#endif
- localNumBackends++;
- }
- }
-
- /* Set the pointer only after completion of a valid table */
- localBackendStatusTable = localtable;
-}
-
-/* ----------
- * pgstat_get_wait_event_type() -
- *
- * Return a string representing the current wait event type, backend is
- * waiting on.
- */
-const char *
-pgstat_get_wait_event_type(uint32 wait_event_info)
-{
- uint32 classId;
- const char *event_type;
-
- /* report process as not waiting. */
- if (wait_event_info == 0)
- return NULL;
-
- classId = wait_event_info & 0xFF000000;
-
- switch (classId)
- {
- case PG_WAIT_LWLOCK:
- event_type = "LWLock";
- break;
- case PG_WAIT_LOCK:
- event_type = "Lock";
- break;
- case PG_WAIT_BUFFER_PIN:
- event_type = "BufferPin";
- break;
- case PG_WAIT_ACTIVITY:
- event_type = "Activity";
- break;
- case PG_WAIT_CLIENT:
- event_type = "Client";
- break;
- case PG_WAIT_EXTENSION:
- event_type = "Extension";
- break;
- case PG_WAIT_IPC:
- event_type = "IPC";
- break;
- case PG_WAIT_TIMEOUT:
- event_type = "Timeout";
- break;
- case PG_WAIT_IO:
- event_type = "IO";
- break;
- default:
- event_type = "???";
- break;
- }
-
- return event_type;
-}
-
-/* ----------
- * pgstat_get_wait_event() -
- *
- * Return a string representing the current wait event, backend is
- * waiting on.
- */
-const char *
-pgstat_get_wait_event(uint32 wait_event_info)
-{
- uint32 classId;
- uint16 eventId;
- const char *event_name;
-
- /* report process as not waiting. */
- if (wait_event_info == 0)
- return NULL;
-
- classId = wait_event_info & 0xFF000000;
- eventId = wait_event_info & 0x0000FFFF;
-
- switch (classId)
- {
- case PG_WAIT_LWLOCK:
- event_name = GetLWLockIdentifier(classId, eventId);
- break;
- case PG_WAIT_LOCK:
- event_name = GetLockNameFromTagType(eventId);
- break;
- case PG_WAIT_BUFFER_PIN:
- event_name = "BufferPin";
- break;
- case PG_WAIT_ACTIVITY:
- {
- WaitEventActivity w = (WaitEventActivity) wait_event_info;
-
- event_name = pgstat_get_wait_activity(w);
- break;
- }
- case PG_WAIT_CLIENT:
- {
- WaitEventClient w = (WaitEventClient) wait_event_info;
-
- event_name = pgstat_get_wait_client(w);
- break;
- }
- case PG_WAIT_EXTENSION:
- event_name = "Extension";
- break;
- case PG_WAIT_IPC:
- {
- WaitEventIPC w = (WaitEventIPC) wait_event_info;
-
- event_name = pgstat_get_wait_ipc(w);
- break;
- }
- case PG_WAIT_TIMEOUT:
- {
- WaitEventTimeout w = (WaitEventTimeout) wait_event_info;
-
- event_name = pgstat_get_wait_timeout(w);
- break;
- }
- case PG_WAIT_IO:
- {
- WaitEventIO w = (WaitEventIO) wait_event_info;
-
- event_name = pgstat_get_wait_io(w);
- break;
- }
- default:
- event_name = "unknown wait event";
- break;
- }
-
- return event_name;
-}
-
-/* ----------
- * pgstat_get_wait_activity() -
- *
- * Convert WaitEventActivity to string.
- * ----------
- */
-static const char *
-pgstat_get_wait_activity(WaitEventActivity w)
-{
- const char *event_name = "unknown wait event";
-
- switch (w)
- {
- case WAIT_EVENT_ARCHIVER_MAIN:
- event_name = "ArchiverMain";
- break;
- case WAIT_EVENT_AUTOVACUUM_MAIN:
- event_name = "AutoVacuumMain";
- break;
- case WAIT_EVENT_BGWRITER_HIBERNATE:
- event_name = "BgWriterHibernate";
- break;
- case WAIT_EVENT_BGWRITER_MAIN:
- event_name = "BgWriterMain";
- break;
- case WAIT_EVENT_CHECKPOINTER_MAIN:
- event_name = "CheckpointerMain";
- break;
- case WAIT_EVENT_LOGICAL_APPLY_MAIN:
- event_name = "LogicalApplyMain";
- break;
- case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
- event_name = "LogicalLauncherMain";
- break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
- case WAIT_EVENT_RECOVERY_WAL_ALL:
- event_name = "RecoveryWalAll";
- break;
- case WAIT_EVENT_RECOVERY_WAL_STREAM:
- event_name = "RecoveryWalStream";
- break;
- case WAIT_EVENT_SYSLOGGER_MAIN:
- event_name = "SysLoggerMain";
- break;
- case WAIT_EVENT_WAL_RECEIVER_MAIN:
- event_name = "WalReceiverMain";
- break;
- case WAIT_EVENT_WAL_SENDER_MAIN:
- event_name = "WalSenderMain";
- break;
- case WAIT_EVENT_WAL_WRITER_MAIN:
- event_name = "WalWriterMain";
- break;
- /* no default case, so that compiler will warn */
- }
-
- return event_name;
-}
-
-/* ----------
- * pgstat_get_wait_client() -
- *
- * Convert WaitEventClient to string.
- * ----------
- */
-static const char *
-pgstat_get_wait_client(WaitEventClient w)
-{
- const char *event_name = "unknown wait event";
-
- switch (w)
- {
- case WAIT_EVENT_CLIENT_READ:
- event_name = "ClientRead";
- break;
- case WAIT_EVENT_CLIENT_WRITE:
- event_name = "ClientWrite";
- break;
- case WAIT_EVENT_LIBPQWALRECEIVER_CONNECT:
- event_name = "LibPQWalReceiverConnect";
- break;
- case WAIT_EVENT_LIBPQWALRECEIVER_RECEIVE:
- event_name = "LibPQWalReceiverReceive";
- break;
- case WAIT_EVENT_SSL_OPEN_SERVER:
- event_name = "SSLOpenServer";
- break;
- case WAIT_EVENT_WAL_RECEIVER_WAIT_START:
- event_name = "WalReceiverWaitStart";
- break;
- case WAIT_EVENT_WAL_SENDER_WAIT_WAL:
- event_name = "WalSenderWaitForWAL";
- break;
- case WAIT_EVENT_WAL_SENDER_WRITE_DATA:
- event_name = "WalSenderWriteData";
- break;
- /* no default case, so that compiler will warn */
- }
-
- return event_name;
-}
-
-/* ----------
- * pgstat_get_wait_ipc() -
- *
- * Convert WaitEventIPC to string.
- * ----------
- */
-static const char *
-pgstat_get_wait_ipc(WaitEventIPC w)
-{
- const char *event_name = "unknown wait event";
-
- switch (w)
- {
- case WAIT_EVENT_BGWORKER_SHUTDOWN:
- event_name = "BgWorkerShutdown";
- break;
- case WAIT_EVENT_BGWORKER_STARTUP:
- event_name = "BgWorkerStartup";
- break;
- case WAIT_EVENT_BTREE_PAGE:
- event_name = "BtreePage";
- break;
- case WAIT_EVENT_CLOG_GROUP_UPDATE:
- event_name = "ClogGroupUpdate";
- break;
- case WAIT_EVENT_EXECUTE_GATHER:
- event_name = "ExecuteGather";
- break;
- case WAIT_EVENT_HASH_BATCH_ALLOCATING:
- event_name = "Hash/Batch/Allocating";
- break;
- case WAIT_EVENT_HASH_BATCH_ELECTING:
- event_name = "Hash/Batch/Electing";
- break;
- case WAIT_EVENT_HASH_BATCH_LOADING:
- event_name = "Hash/Batch/Loading";
- break;
- case WAIT_EVENT_HASH_BUILD_ALLOCATING:
- event_name = "Hash/Build/Allocating";
- break;
- case WAIT_EVENT_HASH_BUILD_ELECTING:
- event_name = "Hash/Build/Electing";
- break;
- case WAIT_EVENT_HASH_BUILD_HASHING_INNER:
- event_name = "Hash/Build/HashingInner";
- break;
- case WAIT_EVENT_HASH_BUILD_HASHING_OUTER:
- event_name = "Hash/Build/HashingOuter";
- break;
- case WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING:
- event_name = "Hash/GrowBatches/Allocating";
- break;
- case WAIT_EVENT_HASH_GROW_BATCHES_DECIDING:
- event_name = "Hash/GrowBatches/Deciding";
- break;
- case WAIT_EVENT_HASH_GROW_BATCHES_ELECTING:
- event_name = "Hash/GrowBatches/Electing";
- break;
- case WAIT_EVENT_HASH_GROW_BATCHES_FINISHING:
- event_name = "Hash/GrowBatches/Finishing";
- break;
- case WAIT_EVENT_HASH_GROW_BATCHES_REPARTITIONING:
- event_name = "Hash/GrowBatches/Repartitioning";
- break;
- case WAIT_EVENT_HASH_GROW_BUCKETS_ALLOCATING:
- event_name = "Hash/GrowBuckets/Allocating";
- break;
- case WAIT_EVENT_HASH_GROW_BUCKETS_ELECTING:
- event_name = "Hash/GrowBuckets/Electing";
- break;
- case WAIT_EVENT_HASH_GROW_BUCKETS_REINSERTING:
- event_name = "Hash/GrowBuckets/Reinserting";
- break;
- case WAIT_EVENT_LOGICAL_SYNC_DATA:
- event_name = "LogicalSyncData";
- break;
- case WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE:
- event_name = "LogicalSyncStateChange";
- break;
- case WAIT_EVENT_MQ_INTERNAL:
- event_name = "MessageQueueInternal";
- break;
- case WAIT_EVENT_MQ_PUT_MESSAGE:
- event_name = "MessageQueuePutMessage";
- break;
- case WAIT_EVENT_MQ_RECEIVE:
- event_name = "MessageQueueReceive";
- break;
- case WAIT_EVENT_MQ_SEND:
- event_name = "MessageQueueSend";
- break;
- case WAIT_EVENT_PARALLEL_BITMAP_SCAN:
- event_name = "ParallelBitmapScan";
- break;
- case WAIT_EVENT_PARALLEL_CREATE_INDEX_SCAN:
- event_name = "ParallelCreateIndexScan";
- break;
- case WAIT_EVENT_PARALLEL_FINISH:
- event_name = "ParallelFinish";
- break;
- case WAIT_EVENT_PROCARRAY_GROUP_UPDATE:
- event_name = "ProcArrayGroupUpdate";
- break;
- case WAIT_EVENT_PROMOTE:
- event_name = "Promote";
- break;
- case WAIT_EVENT_REPLICATION_ORIGIN_DROP:
- event_name = "ReplicationOriginDrop";
- break;
- case WAIT_EVENT_REPLICATION_SLOT_DROP:
- event_name = "ReplicationSlotDrop";
- break;
- case WAIT_EVENT_SAFE_SNAPSHOT:
- event_name = "SafeSnapshot";
- break;
- case WAIT_EVENT_SYNC_REP:
- event_name = "SyncRep";
- break;
- /* no default case, so that compiler will warn */
- }
-
- return event_name;
-}
-
-/* ----------
- * pgstat_get_wait_timeout() -
- *
- * Convert WaitEventTimeout to string.
- * ----------
- */
-static const char *
-pgstat_get_wait_timeout(WaitEventTimeout w)
-{
- const char *event_name = "unknown wait event";
-
- switch (w)
- {
- case WAIT_EVENT_BASE_BACKUP_THROTTLE:
- event_name = "BaseBackupThrottle";
- break;
- case WAIT_EVENT_PG_SLEEP:
- event_name = "PgSleep";
- break;
- case WAIT_EVENT_RECOVERY_APPLY_DELAY:
- event_name = "RecoveryApplyDelay";
- break;
- /* no default case, so that compiler will warn */
- }
-
- return event_name;
-}
-
-/* ----------
- * pgstat_get_wait_io() -
- *
- * Convert WaitEventIO to string.
- * ----------
- */
-static const char *
-pgstat_get_wait_io(WaitEventIO w)
-{
- const char *event_name = "unknown wait event";
-
- switch (w)
- {
- case WAIT_EVENT_BUFFILE_READ:
- event_name = "BufFileRead";
- break;
- case WAIT_EVENT_BUFFILE_WRITE:
- event_name = "BufFileWrite";
- break;
- case WAIT_EVENT_CONTROL_FILE_READ:
- event_name = "ControlFileRead";
- break;
- case WAIT_EVENT_CONTROL_FILE_SYNC:
- event_name = "ControlFileSync";
- break;
- case WAIT_EVENT_CONTROL_FILE_SYNC_UPDATE:
- event_name = "ControlFileSyncUpdate";
- break;
- case WAIT_EVENT_CONTROL_FILE_WRITE:
- event_name = "ControlFileWrite";
- break;
- case WAIT_EVENT_CONTROL_FILE_WRITE_UPDATE:
- event_name = "ControlFileWriteUpdate";
- break;
- case WAIT_EVENT_COPY_FILE_READ:
- event_name = "CopyFileRead";
- break;
- case WAIT_EVENT_COPY_FILE_WRITE:
- event_name = "CopyFileWrite";
- break;
- case WAIT_EVENT_DATA_FILE_EXTEND:
- event_name = "DataFileExtend";
- break;
- case WAIT_EVENT_DATA_FILE_FLUSH:
- event_name = "DataFileFlush";
- break;
- case WAIT_EVENT_DATA_FILE_IMMEDIATE_SYNC:
- event_name = "DataFileImmediateSync";
- break;
- case WAIT_EVENT_DATA_FILE_PREFETCH:
- event_name = "DataFilePrefetch";
- break;
- case WAIT_EVENT_DATA_FILE_READ:
- event_name = "DataFileRead";
- break;
- case WAIT_EVENT_DATA_FILE_SYNC:
- event_name = "DataFileSync";
- break;
- case WAIT_EVENT_DATA_FILE_TRUNCATE:
- event_name = "DataFileTruncate";
- break;
- case WAIT_EVENT_DATA_FILE_WRITE:
- event_name = "DataFileWrite";
- break;
- case WAIT_EVENT_DSM_FILL_ZERO_WRITE:
- event_name = "DSMFillZeroWrite";
- break;
- case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ:
- event_name = "LockFileAddToDataDirRead";
- break;
- case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC:
- event_name = "LockFileAddToDataDirSync";
- break;
- case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE:
- event_name = "LockFileAddToDataDirWrite";
- break;
- case WAIT_EVENT_LOCK_FILE_CREATE_READ:
- event_name = "LockFileCreateRead";
- break;
- case WAIT_EVENT_LOCK_FILE_CREATE_SYNC:
- event_name = "LockFileCreateSync";
- break;
- case WAIT_EVENT_LOCK_FILE_CREATE_WRITE:
- event_name = "LockFileCreateWrite";
- break;
- case WAIT_EVENT_LOCK_FILE_RECHECKDATADIR_READ:
- event_name = "LockFileReCheckDataDirRead";
- break;
- case WAIT_EVENT_LOGICAL_REWRITE_CHECKPOINT_SYNC:
- event_name = "LogicalRewriteCheckpointSync";
- break;
- case WAIT_EVENT_LOGICAL_REWRITE_MAPPING_SYNC:
- event_name = "LogicalRewriteMappingSync";
- break;
- case WAIT_EVENT_LOGICAL_REWRITE_MAPPING_WRITE:
- event_name = "LogicalRewriteMappingWrite";
- break;
- case WAIT_EVENT_LOGICAL_REWRITE_SYNC:
- event_name = "LogicalRewriteSync";
- break;
- case WAIT_EVENT_LOGICAL_REWRITE_TRUNCATE:
- event_name = "LogicalRewriteTruncate";
- break;
- case WAIT_EVENT_LOGICAL_REWRITE_WRITE:
- event_name = "LogicalRewriteWrite";
- break;
- case WAIT_EVENT_RELATION_MAP_READ:
- event_name = "RelationMapRead";
- break;
- case WAIT_EVENT_RELATION_MAP_SYNC:
- event_name = "RelationMapSync";
- break;
- case WAIT_EVENT_RELATION_MAP_WRITE:
- event_name = "RelationMapWrite";
- break;
- case WAIT_EVENT_REORDER_BUFFER_READ:
- event_name = "ReorderBufferRead";
- break;
- case WAIT_EVENT_REORDER_BUFFER_WRITE:
- event_name = "ReorderBufferWrite";
- break;
- case WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ:
- event_name = "ReorderLogicalMappingRead";
- break;
- case WAIT_EVENT_REPLICATION_SLOT_READ:
- event_name = "ReplicationSlotRead";
- break;
- case WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC:
- event_name = "ReplicationSlotRestoreSync";
- break;
- case WAIT_EVENT_REPLICATION_SLOT_SYNC:
- event_name = "ReplicationSlotSync";
- break;
- case WAIT_EVENT_REPLICATION_SLOT_WRITE:
- event_name = "ReplicationSlotWrite";
- break;
- case WAIT_EVENT_SLRU_FLUSH_SYNC:
- event_name = "SLRUFlushSync";
- break;
- case WAIT_EVENT_SLRU_READ:
- event_name = "SLRURead";
- break;
- case WAIT_EVENT_SLRU_SYNC:
- event_name = "SLRUSync";
- break;
- case WAIT_EVENT_SLRU_WRITE:
- event_name = "SLRUWrite";
- break;
- case WAIT_EVENT_SNAPBUILD_READ:
- event_name = "SnapbuildRead";
- break;
- case WAIT_EVENT_SNAPBUILD_SYNC:
- event_name = "SnapbuildSync";
- break;
- case WAIT_EVENT_SNAPBUILD_WRITE:
- event_name = "SnapbuildWrite";
- break;
- case WAIT_EVENT_TIMELINE_HISTORY_FILE_SYNC:
- event_name = "TimelineHistoryFileSync";
- break;
- case WAIT_EVENT_TIMELINE_HISTORY_FILE_WRITE:
- event_name = "TimelineHistoryFileWrite";
- break;
- case WAIT_EVENT_TIMELINE_HISTORY_READ:
- event_name = "TimelineHistoryRead";
- break;
- case WAIT_EVENT_TIMELINE_HISTORY_SYNC:
- event_name = "TimelineHistorySync";
- break;
- case WAIT_EVENT_TIMELINE_HISTORY_WRITE:
- event_name = "TimelineHistoryWrite";
- break;
- case WAIT_EVENT_TWOPHASE_FILE_READ:
- event_name = "TwophaseFileRead";
- break;
- case WAIT_EVENT_TWOPHASE_FILE_SYNC:
- event_name = "TwophaseFileSync";
- break;
- case WAIT_EVENT_TWOPHASE_FILE_WRITE:
- event_name = "TwophaseFileWrite";
- break;
- case WAIT_EVENT_WALSENDER_TIMELINE_HISTORY_READ:
- event_name = "WALSenderTimelineHistoryRead";
- break;
- case WAIT_EVENT_WAL_BOOTSTRAP_SYNC:
- event_name = "WALBootstrapSync";
- break;
- case WAIT_EVENT_WAL_BOOTSTRAP_WRITE:
- event_name = "WALBootstrapWrite";
- break;
- case WAIT_EVENT_WAL_COPY_READ:
- event_name = "WALCopyRead";
- break;
- case WAIT_EVENT_WAL_COPY_SYNC:
- event_name = "WALCopySync";
- break;
- case WAIT_EVENT_WAL_COPY_WRITE:
- event_name = "WALCopyWrite";
- break;
- case WAIT_EVENT_WAL_INIT_SYNC:
- event_name = "WALInitSync";
- break;
- case WAIT_EVENT_WAL_INIT_WRITE:
- event_name = "WALInitWrite";
- break;
- case WAIT_EVENT_WAL_READ:
- event_name = "WALRead";
- break;
- case WAIT_EVENT_WAL_SYNC:
- event_name = "WALSync";
- break;
- case WAIT_EVENT_WAL_SYNC_METHOD_ASSIGN:
- event_name = "WALSyncMethodAssign";
- break;
- case WAIT_EVENT_WAL_WRITE:
- event_name = "WALWrite";
- break;
-
- /* no default case, so that compiler will warn */
- }
-
- return event_name;
-}
-
-
-/* ----------
- * pgstat_get_backend_current_activity() -
- *
- * Return a string representing the current activity of the backend with
- * the specified PID. This looks directly at the BackendStatusArray,
- * and so will provide current information regardless of the age of our
- * transaction's snapshot of the status array.
- *
- * It is the caller's responsibility to invoke this only for backends whose
- * state is expected to remain stable while the result is in use. The
- * only current use is in deadlock reporting, where we can expect that
- * the target backend is blocked on a lock. (There are corner cases
- * where the target's wait could get aborted while we are looking at it,
- * but the very worst consequence is to return a pointer to a string
- * that's been changed, so we won't worry too much.)
- *
- * Note: return strings for special cases match pg_stat_get_backend_activity.
- * ----------
- */
-const char *
-pgstat_get_backend_current_activity(int pid, bool checkUser)
-{
- PgBackendStatus *beentry;
- int i;
-
- beentry = BackendStatusArray;
- for (i = 1; i <= MaxBackends; i++)
- {
- /*
- * Although we expect the target backend's entry to be stable, that
- * doesn't imply that anyone else's is. To avoid identifying the
- * wrong backend, while we check for a match to the desired PID we
- * must follow the protocol of retrying if st_changecount changes
- * while we examine the entry, or if it's odd. (This might be
- * unnecessary, since fetching or storing an int is almost certainly
- * atomic, but let's play it safe.) We use a volatile pointer here to
- * ensure the compiler doesn't try to get cute.
- */
- volatile PgBackendStatus *vbeentry = beentry;
- bool found;
-
- for (;;)
- {
- int before_changecount;
- int after_changecount;
-
- pgstat_save_changecount_before(vbeentry, before_changecount);
-
- found = (vbeentry->st_procpid == pid);
-
- pgstat_save_changecount_after(vbeentry, after_changecount);
-
- if (before_changecount == after_changecount &&
- (before_changecount & 1) == 0)
- break;
-
- /* Make sure we can break out of loop if stuck... */
- CHECK_FOR_INTERRUPTS();
- }
-
- if (found)
- {
- /* Now it is safe to use the non-volatile pointer */
- if (checkUser && !superuser() && beentry->st_userid != GetUserId())
- return "<insufficient privilege>";
- else if (*(beentry->st_activity_raw) == '\0')
- return "<command string not enabled>";
- else
- {
- /* this'll leak a bit of memory, but that seems acceptable */
- return pgstat_clip_activity(beentry->st_activity_raw);
- }
- }
-
- beentry++;
- }
-
- /* If we get here, caller is in error ... */
- return "<backend information not available>";
-}
-
-/* ----------
- * pgstat_get_crashed_backend_activity() -
- *
- * Return a string representing the current activity of the backend with
- * the specified PID. Like the function above, but reads shared memory with
- * the expectation that it may be corrupt. On success, copy the string
- * into the "buffer" argument and return that pointer. On failure,
- * return NULL.
- *
- * This function is only intended to be used by the postmaster to report the
- * query that crashed a backend. In particular, no attempt is made to
- * follow the correct concurrency protocol when accessing the
- * BackendStatusArray. But that's OK, in the worst case we'll return a
- * corrupted message. We also must take care not to trip on ereport(ERROR).
- * ----------
- */
-const char *
-pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
-{
- volatile PgBackendStatus *beentry;
- int i;
-
- beentry = BackendStatusArray;
-
- /*
- * We probably shouldn't get here before shared memory has been set up,
- * but be safe.
- */
- if (beentry == NULL || BackendActivityBuffer == NULL)
- return NULL;
-
- for (i = 1; i <= MaxBackends; i++)
- {
- if (beentry->st_procpid == pid)
- {
- /* Read pointer just once, so it can't change after validation */
- const char *activity = beentry->st_activity_raw;
- const char *activity_last;
-
- /*
- * We mustn't access activity string before we verify that it
- * falls within the BackendActivityBuffer. To make sure that the
- * entire string including its ending is contained within the
- * buffer, subtract one activity length from the buffer size.
- */
- activity_last = BackendActivityBuffer + BackendActivityBufferSize
- - pgstat_track_activity_query_size;
-
- if (activity < BackendActivityBuffer ||
- activity > activity_last)
- return NULL;
-
- /* If no string available, no point in a report */
- if (activity[0] == '\0')
- return NULL;
-
- /*
- * Copy only ASCII-safe characters so we don't run into encoding
- * problems when reporting the message; and be sure not to run off
- * the end of memory. As only ASCII characters are reported, it
- * doesn't seem necessary to perform multibyte aware clipping.
- */
- ascii_safe_strlcpy(buffer, activity,
- Min(buflen, pgstat_track_activity_query_size));
-
- return buffer;
- }
-
- beentry++;
- }
-
- /* PID not found */
- return NULL;
-}
-
-const char *
-pgstat_get_backend_desc(BackendType backendType)
-{
- const char *backendDesc = "unknown process type";
-
- switch (backendType)
- {
- case B_AUTOVAC_LAUNCHER:
- backendDesc = "autovacuum launcher";
- break;
- case B_AUTOVAC_WORKER:
- backendDesc = "autovacuum worker";
- break;
- case B_BACKEND:
- backendDesc = "client backend";
- break;
- case B_BG_WORKER:
- backendDesc = "background worker";
- break;
- case B_BG_WRITER:
- backendDesc = "background writer";
- break;
- case B_ARCHIVER:
- backendDesc = "archiver";
- break;
- case B_CHECKPOINTER:
- backendDesc = "checkpointer";
- break;
- case B_STARTUP:
- backendDesc = "startup";
- break;
- case B_WAL_RECEIVER:
- backendDesc = "walreceiver";
- break;
- case B_WAL_SENDER:
- backendDesc = "walsender";
- break;
- case B_WAL_WRITER:
- backendDesc = "walwriter";
- break;
- }
-
- return backendDesc;
-}
-
-/* ------------------------------------------------------------
- * Local support functions follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * pgstat_setheader() -
- *
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
- * ----------
- */
-void
-pgstat_send_archiver(const char *xlog, bool failed)
-{
- PgStat_MsgArchiver msg;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* ----------
- * pgstat_send_bgwriter() -
- *
- * Send bgwriter statistics to the collector
- * ----------
- */
-void
-pgstat_send_bgwriter(void)
-{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
-
- /*
- * This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
- */
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
- return;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
-
- /*
- * Clear out the statistics buffer, so it can be re-used.
- */
- MemSet(&BgWriterStats, 0, sizeof(BgWriterStats));
-}
-
-
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, pgstat_sighup_handler);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, pgstat_exit);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- /*
- * Identify myself via ps
- */
- init_ps_display("stats collector", "", "", "");
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do recognize got_SIGHUP inside
- * the inner loop, which means that such interrupts will get serviced but
- * the latch won't get cleared until next time there is a break in the
- * action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (need_exit)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * need_exit becomes set.
- */
- while (!need_exit)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (got_SIGHUP)
- {
- got_SIGHUP = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry((PgStat_MsgInquiry *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat((PgStat_MsgTabstat *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge((PgStat_MsgTabpurge *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb((PgStat_MsgDropdb *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter((PgStat_MsgResetcounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(
- (PgStat_MsgResetsharedcounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(
- (PgStat_MsgResetsinglecounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac((PgStat_MsgAutovacStart *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum((PgStat_MsgVacuum *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze((PgStat_MsgAnalyze *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver((PgStat_MsgArchiver *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter((PgStat_MsgBgWriter *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat((PgStat_MsgFuncstat *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge((PgStat_MsgFuncpurge *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict((PgStat_MsgRecoveryConflict *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock((PgStat_MsgDeadlock *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile((PgStat_MsgTempFile *) &msg, len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- exit(0);
-}
-
-
-/* SIGQUIT signal handler for collector process */
-static void
-pgstat_exit(SIGNAL_ARGS)
-{
- int save_errno = errno;
-
- need_exit = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
-}
-
-/* SIGHUP handler for collector process */
-static void
-pgstat_sighup_handler(SIGNAL_ARGS)
-{
- int save_errno = errno;
-
- got_SIGHUP = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
-{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
-
- return result;
-}
-
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
-{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
-}
-
-
-/* ----------
- * pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
- * ----------
- */
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
-{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- FILE *fpout;
- int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
- int rc;
-
- elog(DEBUG2, "writing stats file \"%s\"", statfile);
-
- /*
- * Open the statistics temp file to write out the current values.
- */
- fpout = AllocateFile(tmpfile, PG_BINARY_W);
- if (fpout == NULL)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not open temporary statistics file \"%s\": %m",
- tmpfile)));
- return;
- }
-
- /*
- * Set the timestamp of the stats file.
- */
- globalStats.stats_timestamp = GetCurrentTimestamp();
-
- /*
- * Write the file header --- currently just a format ID.
- */
- format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Write global stats struct
- */
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Write archiver stats struct
- */
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Walk through the database table.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- /*
- * Write out the table and function stats for this DB into the
- * appropriate per-DB stat file, if required.
- */
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
-
- pgstat_write_db_statsfile(dbentry, permanent);
- }
-
- /*
- * Write out the DB entry. We don't write the tables or functions
- * pointers, since they're of no use to any other process.
- */
- fputc('D', fpout);
- rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
- */
- fputc('E', fpout);
-
- if (ferror(fpout))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write temporary statistics file \"%s\": %m",
- tmpfile)));
- FreeFile(fpout);
- unlink(tmpfile);
- }
- else if (FreeFile(fpout) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not close temporary statistics file \"%s\": %m",
- tmpfile)));
- unlink(tmpfile);
- }
- else if (rename(tmpfile, statfile) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
- tmpfile, statfile)));
- unlink(tmpfile);
- }
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
-}
-
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
-
-/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
- * ----------
- */
-static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
-{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpout;
- int32 format_id;
- Oid dbid = dbentry->databaseid;
- int rc;
- char tmpfile[MAXPGPATH];
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "writing stats file \"%s\"", statfile);
-
- /*
- * Open the statistics temp file to write out the current values.
- */
- fpout = AllocateFile(tmpfile, PG_BINARY_W);
- if (fpout == NULL)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not open temporary statistics file \"%s\": %m",
- tmpfile)));
- return;
- }
-
- /*
- * Write the file header --- currently just a format ID.
- */
- format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Walk through the database's access stats per table.
- */
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
- {
- fputc('T', fpout);
- rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * Walk through the database's function stats table.
- */
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
- {
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
- */
- fputc('E', fpout);
-
- if (ferror(fpout))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write temporary statistics file \"%s\": %m",
- tmpfile)));
- FreeFile(fpout);
- unlink(tmpfile);
- }
- else if (FreeFile(fpout) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not close temporary statistics file \"%s\": %m",
- tmpfile)));
- unlink(tmpfile);
- }
- else if (rename(tmpfile, statfile) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
- tmpfile, statfile)));
- unlink(tmpfile);
- }
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_statsfiles() -
- *
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
- *
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
- * ----------
- */
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
- FILE *fpin;
- int32 format_id;
- bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
-
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
-
- /*
- * Set the current timestamp (will be kept only in case we can't load an
- * existing statsfile).
- */
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
- *
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
- * condition is suspicious.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return dbhash;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- goto done;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
- goto done;
- }
-
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
- /*
- * Read archiver stats struct
- */
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
- goto done;
- }
-
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Add to the DB hash
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-
- return dbhash;
-}
-
-
-/* ----------
- * pgstat_read_db_statsfile() -
- *
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
- * ----------
- */
-static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
-{
- PgStat_StatTabEntry *tabentry;
- PgStat_StatTabEntry tabbuf;
- PgStat_StatFuncEntry funcbuf;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpin;
- int32 format_id;
- bool found;
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
-
- /*
- * Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
- *
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
- * condition is suspicious.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- goto done;
- }
-
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'T' A PgStat_StatTabEntry follows.
- */
- case 'T':
- if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
- fpin) != sizeof(PgStat_StatTabEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(tabentry, &tabbuf, sizeof(tabbuf));
- break;
-
- /*
- * 'F' A PgStat_StatFuncEntry follows.
- */
- case 'F':
- if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
- fpin) != sizeof(PgStat_StatFuncEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(funcentry, &funcbuf, sizeof(funcbuf));
- break;
-
- /*
- * 'E' The EOF marker of a complete stats file.
- */
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
-}
-
-
-/* ----------
- * pgstat_setup_memcxt() -
- *
- * Create pgStatLocalContext, if not already done.
- * ----------
- */
-static void
-pgstat_setup_memcxt(void)
-{
- if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
-}
-
-
-/* ----------
- * pgstat_clear_snapshot() -
- *
- * Discard any data collected in the current transaction. Any subsequent
- * request will cause new snapshots to be read.
- *
- * This is also invoked during transaction commit or abort to discard
- * the no-longer-wanted snapshot.
- * ----------
- */
-void
-pgstat_clear_snapshot(void)
-{
- /* Release memory, if any was allocated */
- if (pgStatLocalContext)
- MemoryContextDelete(pgStatLocalContext);
-
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetshared() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
-}
-
-/*
- * Convert a potentially unsafely truncated activity string (see
- * PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
- * one.
- *
- * The returned string is allocated in the caller's memory context and may be
- * freed.
- */
-char *
-pgstat_clip_activity(const char *raw_activity)
-{
- char *activity;
- int rawlen;
- int cliplen;
-
- /*
- * Some callers, like pgstat_get_backend_current_activity(), do not
- * guarantee that the buffer isn't concurrently modified. We try to take
- * care that the buffer is always terminated by a NUL byte regardless, but
- * let's still be paranoid about the string's length. In those cases the
- * underlying buffer is guaranteed to be pgstat_track_activity_query_size
- * large.
- */
- activity = pnstrdup(raw_activity, pgstat_track_activity_query_size - 1);
-
- /* now double-guaranteed to be NUL terminated */
- rawlen = strlen(activity);
-
- /*
- * All supported server-encodings make it possible to determine the length
- * of a multi-byte character from its first byte (this is not the case for
- * client encodings, see GB18030). As st_activity is always stored using
- * server encoding, this allows us to perform multi-byte aware truncation,
- * even if the string earlier was truncated in the middle of a multi-byte
- * character.
- */
- cliplen = pg_mbcliplen(activity, rawlen,
- pgstat_track_activity_query_size - 1);
-
- activity[cliplen] = '\0';
-
- return activity;
-}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a663a62fd5..a01b81a594 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -95,6 +95,7 @@
#include "access/transam.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "bootstrap/bootstrap.h"
#include "catalog/pg_control.h"
#include "common/file_perm.h"
@@ -255,7 +256,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -502,7 +502,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1302,12 +1301,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1756,11 +1749,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2595,8 +2583,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -2927,8 +2913,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -2995,13 +2979,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3076,22 +3053,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3550,22 +3511,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3761,8 +3706,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3801,8 +3744,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4003,8 +3945,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -4977,18 +4917,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5101,12 +5029,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5976,7 +5898,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6029,8 +5950,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6262,7 +6181,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index d1ea46deb8..3accdf7bcf 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -31,11 +31,11 @@
#include <sys/stat.h>
#include <sys/time.h>
+#include "bestatus.h"
#include "lib/stringinfo.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/pg_list.h"
-#include "pgstat.h"
#include "pgtime.h"
#include "postmaster/fork_process.h"
#include "postmaster/postmaster.h"
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index a6fdba3f41..0de04159d5 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -45,9 +45,9 @@
#include <unistd.h>
#include "access/xlog.h"
+#include "bestatus.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "postmaster/walwriter.h"
#include "storage/bufmgr.h"
#include "storage/condition_variable.h"
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index def6c03dd0..e30b2dbcf0 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -17,6 +17,7 @@
#include <time.h>
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
+#include "bestatus.h"
#include "catalog/pg_type.h"
#include "common/file_perm.h"
#include "lib/stringinfo.h"
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7027737e67..75a3208f74 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -22,11 +22,11 @@
#include "libpq-fe.h"
#include "pqexpbuffer.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "replication/walreceiver.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index 55b91b5e12..ea1c7e643e 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -19,7 +19,7 @@
#include "funcapi.h"
#include "miscadmin.h"
-#include "pgstat.h"
+#include "bestatus.h"
#include "access/heapam.h"
#include "access/htup.h"
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index dad2b3d065..dbb7c57ebc 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -77,13 +77,12 @@
#include "access/htup_details.h"
#include "access/table.h"
#include "access/xact.h"
-
+#include "bestatus.h"
#include "catalog/indexing.h"
#include "nodes/execnodes.h"
#include "replication/origin.h"
#include "replication/logical.h"
-#include "pgstat.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index a49e226967..607c7ebc24 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -62,10 +62,10 @@
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "catalog/catalog.h"
#include "lib/binaryheap.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index ad44b2bf43..1b792f6626 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -126,7 +126,7 @@
#include "access/transam.h"
#include "access/xact.h"
-#include "pgstat.h"
+#include "bestatus.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 28f5fc23aa..58c94794bc 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -86,26 +86,28 @@
#include "postgres.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "access/table.h"
#include "access/xact.h"
+#include "bestatus.h"
+
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_type.h"
#include "commands/copy.h"
#include "parser/parse_relation.h"
+#include "pgstat.h"
#include "replication/logicallauncher.h"
#include "replication/logicalrelation.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
-#include "utils/snapmgr.h"
#include "storage/ipc.h"
+#include "utils/snapmgr.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
@@ -128,7 +130,7 @@ finish_sync_worker(void)
if (IsTransactionState())
{
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(true);
}
/* And flush all writes. */
@@ -144,6 +146,9 @@ finish_sync_worker(void)
/* Find the main apply worker and signal it. */
logicalrep_worker_wakeup(MyLogicalRepWorker->subid, InvalidOid);
+ /* clean up retained statistics */
+ pgstat_update_stat(true);
+
/* Stop gracefully */
proc_exit(0);
}
@@ -525,7 +530,7 @@ process_syncing_tables_for_apply(XLogRecPtr current_lsn)
if (started_tx)
{
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
}
}
@@ -863,7 +868,7 @@ LogicalRepSyncTableStart(XLogRecPtr *origin_startpos)
MyLogicalRepWorker->relstate,
MyLogicalRepWorker->relstate_lsn);
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
/*
* We want to do the table data sync in a single transaction.
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index f9516515bc..dc675778e3 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -26,6 +26,7 @@
#include "access/table.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "catalog/catalog.h"
#include "catalog/namespace.h"
#include "catalog/pg_subscription.h"
@@ -477,7 +478,7 @@ apply_handle_commit(StringInfo s)
replorigin_session_origin_timestamp = commit_data.committime;
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
store_flush_position(commit_data.end_lsn);
}
@@ -1311,6 +1312,8 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
}
send_feedback(last_received, requestReply, requestReply);
+
+ pgstat_update_stat(false);
}
}
}
diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c
index 33b23b6b6d..c60e69302a 100644
--- a/src/backend/replication/slot.c
+++ b/src/backend/replication/slot.c
@@ -41,9 +41,9 @@
#include "access/transam.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "common/string.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "replication/slot.h"
#include "storage/fd.h"
#include "storage/proc.h"
diff --git a/src/backend/replication/syncrep.c b/src/backend/replication/syncrep.c
index 6c160c13c6..02ec91d98e 100644
--- a/src/backend/replication/syncrep.c
+++ b/src/backend/replication/syncrep.c
@@ -75,8 +75,8 @@
#include <unistd.h>
#include "access/xact.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "replication/syncrep.h"
#include "replication/walsender.h"
#include "replication/walsender_private.h"
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 2e90944ad5..bdca25499d 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -50,6 +50,7 @@
#include "access/timeline.h"
#include "access/transam.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_type.h"
#include "common/ip.h"
@@ -57,7 +58,6 @@
#include "libpq/pqformat.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
#include "storage/ipc.h"
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 9b143f361b..2b38c0c4f5 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -56,6 +56,7 @@
#include "access/xlog_internal.h"
#include "access/xlogutils.h"
+#include "bestatus.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_type.h"
#include "commands/dbcommands.h"
@@ -65,7 +66,6 @@
#include "libpq/pqformat.h"
#include "miscadmin.h"
#include "nodes/replnodes.h"
-#include "pgstat.h"
#include "replication/basebackup.h"
#include "replication/decode.h"
#include "replication/logical.h"
diff --git a/src/backend/statmon/Makefile b/src/backend/statmon/Makefile
new file mode 100644
index 0000000000..64a04878e3
--- /dev/null
+++ b/src/backend/statmon/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for src/backend/statmon
+#
+# IDENTIFICATION
+# src/backend/statmon/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/statmon
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pgstat.o bestatus.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/statmon/bestatus.c b/src/backend/statmon/bestatus.c
new file mode 100644
index 0000000000..292312d05c
--- /dev/null
+++ b/src/backend/statmon/bestatus.c
@@ -0,0 +1,1781 @@
+/* ----------
+ * bestatus.c
+ *
+ * Backend status monitor
+ *
+ * Status data is stored in shared memory. Every backends updates and read it
+ * individually.
+ *
+ * Copyright (c) 2001-2019, PostgreSQL Global Development Group
+ *
+ * src/backend/statmon/bestatus.c
+ * ----------
+ */
+#include "postgres.h"
+
+#include "bestatus.h"
+
+#include "access/xact.h"
+#include "libpq/libpq.h"
+#include "miscadmin.h"
+#include "postmaster/autovacuum.h"
+#include "replication/walsender.h"
+#include "storage/ipc.h"
+#include "storage/lmgr.h"
+#include "storage/sinvaladt.h"
+#include "utils/ascii.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+#include "utils/probes.h"
+
+
+/* Status for backends including auxiliary */
+static LocalPgBackendStatus *localBackendStatusTable = NULL;
+
+/* Total number of backends including auxiliary */
+static int localNumBackends = 0;
+
+/* ----------
+ * Total number of backends including auxiliary
+ *
+ * We reserve a slot for each possible BackendId, plus one for each
+ * possible auxiliary process type. (This scheme assumes there is not
+ * more than one of any auxiliary process type at a time.) MaxBackends
+ * includes autovacuum workers and background workers as well.
+ * ----------
+ */
+#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_activities = false;
+int pgstat_track_activity_query_size = 1024;
+
+static MemoryContext pgBeStatLocalContext = NULL;
+
+/* ------------------------------------------------------------
+ * Functions for management of the shared-memory PgBackendStatus array
+ * ------------------------------------------------------------
+ */
+
+static PgBackendStatus *BackendStatusArray = NULL;
+static PgBackendStatus *MyBEEntry = NULL;
+static char *BackendAppnameBuffer = NULL;
+static char *BackendClientHostnameBuffer = NULL;
+static char *BackendActivityBuffer = NULL;
+static Size BackendActivityBufferSize = 0;
+#ifdef USE_SSL
+static PgBackendSSLStatus *BackendSslStatusBuffer = NULL;
+#endif
+
+static const char *pgstat_get_wait_activity(WaitEventActivity w);
+static const char *pgstat_get_wait_client(WaitEventClient w);
+static const char *pgstat_get_wait_ipc(WaitEventIPC w);
+static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
+static const char *pgstat_get_wait_io(WaitEventIO w);
+static void pgstat_setup_memcxt(void);
+static void bestatus_clear_snapshot(void);
+static void pgstat_beshutdown_hook(int code, Datum arg);
+/*
+ * Report shared-memory space needed by CreateSharedBackendStatus.
+ */
+Size
+BackendStatusShmemSize(void)
+{
+ Size size;
+
+ /* BackendStatusArray: */
+ size = mul_size(sizeof(PgBackendStatus), NumBackendStatSlots);
+ /* BackendAppnameBuffer: */
+ size = add_size(size,
+ mul_size(NAMEDATALEN, NumBackendStatSlots));
+ /* BackendClientHostnameBuffer: */
+ size = add_size(size,
+ mul_size(NAMEDATALEN, NumBackendStatSlots));
+ /* BackendActivityBuffer: */
+ size = add_size(size,
+ mul_size(pgstat_track_activity_query_size, NumBackendStatSlots));
+#ifdef USE_SSL
+ /* BackendSslStatusBuffer: */
+ size = add_size(size,
+ mul_size(sizeof(PgBackendSSLStatus), NumBackendStatSlots));
+#endif
+ return size;
+}
+
+/*
+ * Initialize the shared status array and several string buffers
+ * during postmaster startup.
+ */
+void
+CreateSharedBackendStatus(void)
+{
+ Size size;
+ bool found;
+ int i;
+ char *buffer;
+
+ /* Create or attach to the shared array */
+ size = mul_size(sizeof(PgBackendStatus), NumBackendStatSlots);
+ BackendStatusArray = (PgBackendStatus *)
+ ShmemInitStruct("Backend Status Array", size, &found);
+
+ if (!found)
+ {
+ /*
+ * We're the first - initialize.
+ */
+ MemSet(BackendStatusArray, 0, size);
+ }
+
+ /* Create or attach to the shared appname buffer */
+ size = mul_size(NAMEDATALEN, NumBackendStatSlots);
+ BackendAppnameBuffer = (char *)
+ ShmemInitStruct("Backend Application Name Buffer", size, &found);
+
+ if (!found)
+ {
+ MemSet(BackendAppnameBuffer, 0, size);
+
+ /* Initialize st_appname pointers. */
+ buffer = BackendAppnameBuffer;
+ for (i = 0; i < NumBackendStatSlots; i++)
+ {
+ BackendStatusArray[i].st_appname = buffer;
+ buffer += NAMEDATALEN;
+ }
+ }
+
+ /* Create or attach to the shared client hostname buffer */
+ size = mul_size(NAMEDATALEN, NumBackendStatSlots);
+ BackendClientHostnameBuffer = (char *)
+ ShmemInitStruct("Backend Client Host Name Buffer", size, &found);
+
+ if (!found)
+ {
+ MemSet(BackendClientHostnameBuffer, 0, size);
+
+ /* Initialize st_clienthostname pointers. */
+ buffer = BackendClientHostnameBuffer;
+ for (i = 0; i < NumBackendStatSlots; i++)
+ {
+ BackendStatusArray[i].st_clienthostname = buffer;
+ buffer += NAMEDATALEN;
+ }
+ }
+
+ /* Create or attach to the shared activity buffer */
+ BackendActivityBufferSize = mul_size(pgstat_track_activity_query_size,
+ NumBackendStatSlots);
+ BackendActivityBuffer = (char *)
+ ShmemInitStruct("Backend Activity Buffer",
+ BackendActivityBufferSize,
+ &found);
+
+ if (!found)
+ {
+ MemSet(BackendActivityBuffer, 0, BackendActivityBufferSize);
+
+ /* Initialize st_activity pointers. */
+ buffer = BackendActivityBuffer;
+ for (i = 0; i < NumBackendStatSlots; i++)
+ {
+ BackendStatusArray[i].st_activity_raw = buffer;
+ buffer += pgstat_track_activity_query_size;
+ }
+ }
+
+#ifdef USE_SSL
+ /* Create or attach to the shared SSL status buffer */
+ size = mul_size(sizeof(PgBackendSSLStatus), NumBackendStatSlots);
+ BackendSslStatusBuffer = (PgBackendSSLStatus *)
+ ShmemInitStruct("Backend SSL Status Buffer", size, &found);
+
+ if (!found)
+ {
+ PgBackendSSLStatus *ptr;
+
+ MemSet(BackendSslStatusBuffer, 0, size);
+
+ /* Initialize st_sslstatus pointers. */
+ ptr = BackendSslStatusBuffer;
+ for (i = 0; i < NumBackendStatSlots; i++)
+ {
+ BackendStatusArray[i].st_sslstatus = ptr;
+ ptr++;
+ }
+ }
+#endif
+}
+
+/* ----------
+ * pgstat_bearray_initialize() -
+ *
+ * Initialize pgstats state, and set up our on-proc-exit hook.
+ * Called from InitPostgres and AuxiliaryProcessMain. For auxiliary process,
+ * MyBackendId is invalid. Otherwise, MyBackendId must be set,
+ * but we must not have started any transaction yet (since the
+ * exit hook must run after the last transaction exit).
+ * NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
+ * ----------
+ */
+void
+pgstat_bearray_initialize(void)
+{
+ /* Initialize MyBEEntry */
+ if (MyBackendId != InvalidBackendId)
+ {
+ Assert(MyBackendId >= 1 && MyBackendId <= MaxBackends);
+ MyBEEntry = &BackendStatusArray[MyBackendId - 1];
+ }
+ else
+ {
+ /* Must be an auxiliary process */
+ Assert(MyAuxProcType != NotAnAuxProcess);
+
+ /*
+ * Assign the MyBEEntry for an auxiliary process. Since it doesn't
+ * have a BackendId, the slot is statically allocated based on the
+ * auxiliary process type (MyAuxProcType). Backends use slots indexed
+ * in the range from 1 to MaxBackends (inclusive), so we use
+ * MaxBackends + AuxBackendType + 1 as the index of the slot for an
+ * auxiliary process.
+ */
+ MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
+ }
+
+ /* Set up a process-exit hook to clean up */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
+}
+
+/*
+ * Shut down a single backend's statistics reporting at process exit.
+ *
+ * Flush any remaining statistics counts out to the collector.
+ * Without this, operations triggered during backend exit (such as
+ * temp table deletions) won't be counted.
+ *
+ * Lastly, clear out our entry in the PgBackendStatus array.
+ */
+static void
+pgstat_beshutdown_hook(int code, Datum arg)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * Clear my status entry, following the protocol of bumping st_changecount
+ * before and after. We use a volatile pointer here to ensure the
+ * compiler doesn't try to get cute.
+ */
+ pgstat_increment_changecount_before(beentry);
+
+ beentry->st_procpid = 0; /* mark invalid */
+
+ pgstat_increment_changecount_after(beentry);
+}
+
+
+/* ----------
+ * pgstat_bestart() -
+ *
+ * Initialize this backend's entry in the PgBackendStatus array.
+ * Called from InitPostgres.
+ *
+ * Apart from auxiliary processes, MyBackendId, MyDatabaseId,
+ * session userid, and application_name must be set for a
+ * backend (hence, this cannot be combined with pgstat_initialize).
+ * ----------
+ */
+void
+pgstat_bestart(void)
+{
+ SockAddr clientaddr;
+ volatile PgBackendStatus *beentry;
+
+ /*
+ * To minimize the time spent modifying the PgBackendStatus entry, fetch
+ * all the needed data first.
+ */
+
+ /*
+ * We may not have a MyProcPort (eg, if this is the autovacuum process).
+ * If so, use all-zeroes client address, which is dealt with specially in
+ * pg_stat_get_backend_client_addr and pg_stat_get_backend_client_port.
+ */
+ if (MyProcPort)
+ memcpy(&clientaddr, &MyProcPort->raddr, sizeof(clientaddr));
+ else
+ MemSet(&clientaddr, 0, sizeof(clientaddr));
+
+ /*
+ * Initialize my status entry, following the protocol of bumping
+ * st_changecount before and after; and make sure it's even afterwards. We
+ * use a volatile pointer here to ensure the compiler doesn't try to get
+ * cute.
+ */
+ beentry = MyBEEntry;
+
+ /* pgstats state must be initialized from pgstat_initialize() */
+ Assert(beentry != NULL);
+
+ if (MyBackendId != InvalidBackendId)
+ {
+ if (IsAutoVacuumLauncherProcess())
+ {
+ /* Autovacuum Launcher */
+ beentry->st_backendType = B_AUTOVAC_LAUNCHER;
+ }
+ else if (IsAutoVacuumWorkerProcess())
+ {
+ /* Autovacuum Worker */
+ beentry->st_backendType = B_AUTOVAC_WORKER;
+ }
+ else if (am_walsender)
+ {
+ /* Wal sender */
+ beentry->st_backendType = B_WAL_SENDER;
+ }
+ else if (IsBackgroundWorker)
+ {
+ /* bgworker */
+ beentry->st_backendType = B_BG_WORKER;
+ }
+ else
+ {
+ /* client-backend */
+ beentry->st_backendType = B_BACKEND;
+ }
+ }
+ else
+ {
+ /* Must be an auxiliary process */
+ Assert(MyAuxProcType != NotAnAuxProcess);
+ switch (MyAuxProcType)
+ {
+ case StartupProcess:
+ beentry->st_backendType = B_STARTUP;
+ break;
+ case BgWriterProcess:
+ beentry->st_backendType = B_BG_WRITER;
+ break;
+ case CheckpointerProcess:
+ beentry->st_backendType = B_CHECKPOINTER;
+ break;
+ case WalWriterProcess:
+ beentry->st_backendType = B_WAL_WRITER;
+ break;
+ case WalReceiverProcess:
+ beentry->st_backendType = B_WAL_RECEIVER;
+ break;
+ case ArchiverProcess:
+ beentry->st_backendType = B_ARCHIVER;
+ break;
+ default:
+ elog(FATAL, "unrecognized process type: %d",
+ (int) MyAuxProcType);
+ proc_exit(1);
+ }
+ }
+
+ do
+ {
+ pgstat_increment_changecount_before(beentry);
+ } while ((beentry->st_changecount & 1) == 0);
+
+ beentry->st_procpid = MyProcPid;
+ beentry->st_proc_start_timestamp = MyStartTimestamp;
+ beentry->st_activity_start_timestamp = 0;
+ beentry->st_state_start_timestamp = 0;
+ beentry->st_xact_start_timestamp = 0;
+ beentry->st_databaseid = MyDatabaseId;
+
+ /* We have userid for client-backends, wal-sender and bgworker processes */
+ if (beentry->st_backendType == B_BACKEND
+ || beentry->st_backendType == B_WAL_SENDER
+ || beentry->st_backendType == B_BG_WORKER)
+ beentry->st_userid = GetSessionUserId();
+ else
+ beentry->st_userid = InvalidOid;
+
+ beentry->st_clientaddr = clientaddr;
+ if (MyProcPort && MyProcPort->remote_hostname)
+ strlcpy(beentry->st_clienthostname, MyProcPort->remote_hostname,
+ NAMEDATALEN);
+ else
+ beentry->st_clienthostname[0] = '\0';
+#ifdef USE_SSL
+ if (MyProcPort && MyProcPort->ssl != NULL)
+ {
+ beentry->st_ssl = true;
+ beentry->st_sslstatus->ssl_bits = be_tls_get_cipher_bits(MyProcPort);
+ beentry->st_sslstatus->ssl_compression = be_tls_get_compression(MyProcPort);
+ strlcpy(beentry->st_sslstatus->ssl_version, be_tls_get_version(MyProcPort), NAMEDATALEN);
+ strlcpy(beentry->st_sslstatus->ssl_cipher, be_tls_get_cipher(MyProcPort), NAMEDATALEN);
+ be_tls_get_peer_subject_name(MyProcPort, beentry->st_sslstatus->ssl_client_dn, NAMEDATALEN);
+ be_tls_get_peer_serial(MyProcPort, beentry->st_sslstatus->ssl_client_serial, NAMEDATALEN);
+ be_tls_get_peer_issuer_name(MyProcPort, beentry->st_sslstatus->ssl_issuer_dn, NAMEDATALEN);
+ }
+ else
+ {
+ beentry->st_ssl = false;
+ }
+#else
+ beentry->st_ssl = false;
+#endif
+ beentry->st_state = STATE_UNDEFINED;
+ beentry->st_appname[0] = '\0';
+ beentry->st_activity_raw[0] = '\0';
+ /* Also make sure the last byte in each string area is always 0 */
+ beentry->st_clienthostname[NAMEDATALEN - 1] = '\0';
+ beentry->st_appname[NAMEDATALEN - 1] = '\0';
+ beentry->st_activity_raw[pgstat_track_activity_query_size - 1] = '\0';
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+
+ /*
+ * we don't zero st_progress_param here to save cycles; nobody should
+ * examine it until st_progress_command has been set to something other
+ * than PROGRESS_COMMAND_INVALID
+ */
+
+ pgstat_increment_changecount_after(beentry);
+
+ /* Update app name to current GUC setting */
+ if (application_name)
+ pgstat_report_appname(application_name);
+}
+
+/* ----------
+ * pgstat_read_current_status() -
+ *
+ * Copy the current contents of the PgBackendStatus array to local memory,
+ * if not already done in this transaction.
+ * ----------
+ */
+static void
+pgstat_read_current_status(void)
+{
+ volatile PgBackendStatus *beentry;
+ LocalPgBackendStatus *localtable;
+ LocalPgBackendStatus *localentry;
+ char *localappname,
+ *localclienthostname,
+ *localactivity;
+#ifdef USE_SSL
+ PgBackendSSLStatus *localsslstatus;
+#endif
+ int i;
+
+ Assert(IsUnderPostmaster);
+
+ if (localBackendStatusTable)
+ return; /* already done */
+
+ pgstat_setup_memcxt();
+
+ localtable = (LocalPgBackendStatus *)
+ MemoryContextAlloc(pgBeStatLocalContext,
+ sizeof(LocalPgBackendStatus) * NumBackendStatSlots);
+ localappname = (char *)
+ MemoryContextAlloc(pgBeStatLocalContext,
+ NAMEDATALEN * NumBackendStatSlots);
+ localclienthostname = (char *)
+ MemoryContextAlloc(pgBeStatLocalContext,
+ NAMEDATALEN * NumBackendStatSlots);
+ localactivity = (char *)
+ MemoryContextAlloc(pgBeStatLocalContext,
+ pgstat_track_activity_query_size * NumBackendStatSlots);
+#ifdef USE_SSL
+ localsslstatus = (PgBackendSSLStatus *)
+ MemoryContextAlloc(pgBeStatLocalContext,
+ sizeof(PgBackendSSLStatus) * NumBackendStatSlots);
+#endif
+
+ localNumBackends = 0;
+
+ beentry = BackendStatusArray;
+ localentry = localtable;
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * Follow the protocol of retrying if st_changecount changes while we
+ * copy the entry, or if it's odd. (The check for odd is needed to
+ * cover the case where we are able to completely copy the entry while
+ * the source backend is between increment steps.) We use a volatile
+ * pointer here to ensure the compiler doesn't try to get cute.
+ */
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_save_changecount_before(beentry, before_changecount);
+
+ localentry->backendStatus.st_procpid = beentry->st_procpid;
+ if (localentry->backendStatus.st_procpid > 0)
+ {
+ memcpy(&localentry->backendStatus, (char *) beentry, sizeof(PgBackendStatus));
+
+ /*
+ * strcpy is safe even if the string is modified concurrently,
+ * because there's always a \0 at the end of the buffer.
+ */
+ strcpy(localappname, (char *) beentry->st_appname);
+ localentry->backendStatus.st_appname = localappname;
+ strcpy(localclienthostname, (char *) beentry->st_clienthostname);
+ localentry->backendStatus.st_clienthostname = localclienthostname;
+ strcpy(localactivity, (char *) beentry->st_activity_raw);
+ localentry->backendStatus.st_activity_raw = localactivity;
+ localentry->backendStatus.st_ssl = beentry->st_ssl;
+#ifdef USE_SSL
+ if (beentry->st_ssl)
+ {
+ memcpy(localsslstatus, beentry->st_sslstatus, sizeof(PgBackendSSLStatus));
+ localentry->backendStatus.st_sslstatus = localsslstatus;
+ }
+#endif
+ }
+
+ pgstat_save_changecount_after(beentry, after_changecount);
+ if (before_changecount == after_changecount &&
+ (before_changecount & 1) == 0)
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ beentry++;
+ /* Only valid entries get included into the local array */
+ if (localentry->backendStatus.st_procpid > 0)
+ {
+ BackendIdGetTransactionIds(i,
+ &localentry->backend_xid,
+ &localentry->backend_xmin);
+
+ localentry++;
+ localappname += NAMEDATALEN;
+ localclienthostname += NAMEDATALEN;
+ localactivity += pgstat_track_activity_query_size;
+#ifdef USE_SSL
+ localsslstatus++;
+#endif
+ localNumBackends++;
+ }
+ }
+
+ /* Set the pointer only after completion of a valid table */
+ localBackendStatusTable = localtable;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_beentry() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * our local copy of the current-activity entry for one backend.
+ *
+ * NB: caller is responsible for a check if the user is permitted to see
+ * this info (especially the querystring).
+ * ----------
+ */
+PgBackendStatus *
+pgstat_fetch_stat_beentry(int beid)
+{
+ pgstat_read_current_status();
+
+ if (beid < 1 || beid > localNumBackends)
+ return NULL;
+
+ return &localBackendStatusTable[beid - 1].backendStatus;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_local_beentry() -
+ *
+ * Like pgstat_fetch_stat_beentry() but with locally computed additions (like
+ * xid and xmin values of the backend)
+ *
+ * NB: caller is responsible for a check if the user is permitted to see
+ * this info (especially the querystring).
+ * ----------
+ */
+LocalPgBackendStatus *
+pgstat_fetch_stat_local_beentry(int beid)
+{
+ pgstat_read_current_status();
+
+ if (beid < 1 || beid > localNumBackends)
+ return NULL;
+
+ return &localBackendStatusTable[beid - 1];
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_numbackends() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the maximum current backend id.
+ * ----------
+ */
+int
+pgstat_fetch_stat_numbackends(void)
+{
+ pgstat_read_current_status();
+
+ return localNumBackends;
+}
+
+/* ----------
+ * pgstat_get_wait_event_type() -
+ *
+ * Return a string representing the current wait event type, backend is
+ * waiting on.
+ */
+const char *
+pgstat_get_wait_event_type(uint32 wait_event_info)
+{
+ uint32 classId;
+ const char *event_type;
+
+ /* report process as not waiting. */
+ if (wait_event_info == 0)
+ return NULL;
+
+ classId = wait_event_info & 0xFF000000;
+
+ switch (classId)
+ {
+ case PG_WAIT_LWLOCK:
+ event_type = "LWLock";
+ break;
+ case PG_WAIT_LOCK:
+ event_type = "Lock";
+ break;
+ case PG_WAIT_BUFFER_PIN:
+ event_type = "BufferPin";
+ break;
+ case PG_WAIT_ACTIVITY:
+ event_type = "Activity";
+ break;
+ case PG_WAIT_CLIENT:
+ event_type = "Client";
+ break;
+ case PG_WAIT_EXTENSION:
+ event_type = "Extension";
+ break;
+ case PG_WAIT_IPC:
+ event_type = "IPC";
+ break;
+ case PG_WAIT_TIMEOUT:
+ event_type = "Timeout";
+ break;
+ case PG_WAIT_IO:
+ event_type = "IO";
+ break;
+ default:
+ event_type = "???";
+ break;
+ }
+
+ return event_type;
+}
+
+/* ----------
+ * pgstat_get_wait_event() -
+ *
+ * Return a string representing the current wait event, backend is
+ * waiting on.
+ */
+const char *
+pgstat_get_wait_event(uint32 wait_event_info)
+{
+ uint32 classId;
+ uint16 eventId;
+ const char *event_name;
+
+ /* report process as not waiting. */
+ if (wait_event_info == 0)
+ return NULL;
+
+ classId = wait_event_info & 0xFF000000;
+ eventId = wait_event_info & 0x0000FFFF;
+
+ switch (classId)
+ {
+ case PG_WAIT_LWLOCK:
+ event_name = GetLWLockIdentifier(classId, eventId);
+ break;
+ case PG_WAIT_LOCK:
+ event_name = GetLockNameFromTagType(eventId);
+ break;
+ case PG_WAIT_BUFFER_PIN:
+ event_name = "BufferPin";
+ break;
+ case PG_WAIT_ACTIVITY:
+ {
+ WaitEventActivity w = (WaitEventActivity) wait_event_info;
+
+ event_name = pgstat_get_wait_activity(w);
+ break;
+ }
+ case PG_WAIT_CLIENT:
+ {
+ WaitEventClient w = (WaitEventClient) wait_event_info;
+
+ event_name = pgstat_get_wait_client(w);
+ break;
+ }
+ case PG_WAIT_EXTENSION:
+ event_name = "Extension";
+ break;
+ case PG_WAIT_IPC:
+ {
+ WaitEventIPC w = (WaitEventIPC) wait_event_info;
+
+ event_name = pgstat_get_wait_ipc(w);
+ break;
+ }
+ case PG_WAIT_TIMEOUT:
+ {
+ WaitEventTimeout w = (WaitEventTimeout) wait_event_info;
+
+ event_name = pgstat_get_wait_timeout(w);
+ break;
+ }
+ case PG_WAIT_IO:
+ {
+ WaitEventIO w = (WaitEventIO) wait_event_info;
+
+ event_name = pgstat_get_wait_io(w);
+ break;
+ }
+ default:
+ event_name = "unknown wait event";
+ break;
+ }
+
+ return event_name;
+}
+
+/* ----------
+ * pgstat_get_wait_activity() -
+ *
+ * Convert WaitEventActivity to string.
+ * ----------
+ */
+static const char *
+pgstat_get_wait_activity(WaitEventActivity w)
+{
+ const char *event_name = "unknown wait event";
+
+ switch (w)
+ {
+ case WAIT_EVENT_ARCHIVER_MAIN:
+ event_name = "ArchiverMain";
+ break;
+ case WAIT_EVENT_AUTOVACUUM_MAIN:
+ event_name = "AutoVacuumMain";
+ break;
+ case WAIT_EVENT_BGWRITER_HIBERNATE:
+ event_name = "BgWriterHibernate";
+ break;
+ case WAIT_EVENT_BGWRITER_MAIN:
+ event_name = "BgWriterMain";
+ break;
+ case WAIT_EVENT_CHECKPOINTER_MAIN:
+ event_name = "CheckpointerMain";
+ break;
+ case WAIT_EVENT_LOGICAL_APPLY_MAIN:
+ event_name = "LogicalApplyMain";
+ break;
+ case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
+ event_name = "LogicalLauncherMain";
+ break;
+ case WAIT_EVENT_RECOVERY_WAL_ALL:
+ event_name = "RecoveryWalAll";
+ break;
+ case WAIT_EVENT_RECOVERY_WAL_STREAM:
+ event_name = "RecoveryWalStream";
+ break;
+ case WAIT_EVENT_SYSLOGGER_MAIN:
+ event_name = "SysLoggerMain";
+ break;
+ case WAIT_EVENT_WAL_RECEIVER_MAIN:
+ event_name = "WalReceiverMain";
+ break;
+ case WAIT_EVENT_WAL_SENDER_MAIN:
+ event_name = "WalSenderMain";
+ break;
+ case WAIT_EVENT_WAL_WRITER_MAIN:
+ event_name = "WalWriterMain";
+ break;
+ /* no default case, so that compiler will warn */
+ }
+
+ return event_name;
+}
+
+/* ----------
+ * pgstat_get_wait_client() -
+ *
+ * Convert WaitEventClient to string.
+ * ----------
+ */
+static const char *
+pgstat_get_wait_client(WaitEventClient w)
+{
+ const char *event_name = "unknown wait event";
+
+ switch (w)
+ {
+ case WAIT_EVENT_CLIENT_READ:
+ event_name = "ClientRead";
+ break;
+ case WAIT_EVENT_CLIENT_WRITE:
+ event_name = "ClientWrite";
+ break;
+ case WAIT_EVENT_LIBPQWALRECEIVER_CONNECT:
+ event_name = "LibPQWalReceiverConnect";
+ break;
+ case WAIT_EVENT_LIBPQWALRECEIVER_RECEIVE:
+ event_name = "LibPQWalReceiverReceive";
+ break;
+ case WAIT_EVENT_SSL_OPEN_SERVER:
+ event_name = "SSLOpenServer";
+ break;
+ case WAIT_EVENT_WAL_RECEIVER_WAIT_START:
+ event_name = "WalReceiverWaitStart";
+ break;
+ case WAIT_EVENT_WAL_SENDER_WAIT_WAL:
+ event_name = "WalSenderWaitForWAL";
+ break;
+ case WAIT_EVENT_WAL_SENDER_WRITE_DATA:
+ event_name = "WalSenderWriteData";
+ break;
+ /* no default case, so that compiler will warn */
+ }
+
+ return event_name;
+}
+
+/* ----------
+ * pgstat_get_wait_ipc() -
+ *
+ * Convert WaitEventIPC to string.
+ * ----------
+ */
+static const char *
+pgstat_get_wait_ipc(WaitEventIPC w)
+{
+ const char *event_name = "unknown wait event";
+
+ switch (w)
+ {
+ case WAIT_EVENT_BGWORKER_SHUTDOWN:
+ event_name = "BgWorkerShutdown";
+ break;
+ case WAIT_EVENT_BGWORKER_STARTUP:
+ event_name = "BgWorkerStartup";
+ break;
+ case WAIT_EVENT_BTREE_PAGE:
+ event_name = "BtreePage";
+ break;
+ case WAIT_EVENT_CLOG_GROUP_UPDATE:
+ event_name = "ClogGroupUpdate";
+ break;
+ case WAIT_EVENT_EXECUTE_GATHER:
+ event_name = "ExecuteGather";
+ break;
+ case WAIT_EVENT_HASH_BATCH_ALLOCATING:
+ event_name = "Hash/Batch/Allocating";
+ break;
+ case WAIT_EVENT_HASH_BATCH_ELECTING:
+ event_name = "Hash/Batch/Electing";
+ break;
+ case WAIT_EVENT_HASH_BATCH_LOADING:
+ event_name = "Hash/Batch/Loading";
+ break;
+ case WAIT_EVENT_HASH_BUILD_ALLOCATING:
+ event_name = "Hash/Build/Allocating";
+ break;
+ case WAIT_EVENT_HASH_BUILD_ELECTING:
+ event_name = "Hash/Build/Electing";
+ break;
+ case WAIT_EVENT_HASH_BUILD_HASHING_INNER:
+ event_name = "Hash/Build/HashingInner";
+ break;
+ case WAIT_EVENT_HASH_BUILD_HASHING_OUTER:
+ event_name = "Hash/Build/HashingOuter";
+ break;
+ case WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING:
+ event_name = "Hash/GrowBatches/Allocating";
+ break;
+ case WAIT_EVENT_HASH_GROW_BATCHES_DECIDING:
+ event_name = "Hash/GrowBatches/Deciding";
+ break;
+ case WAIT_EVENT_HASH_GROW_BATCHES_ELECTING:
+ event_name = "Hash/GrowBatches/Electing";
+ break;
+ case WAIT_EVENT_HASH_GROW_BATCHES_FINISHING:
+ event_name = "Hash/GrowBatches/Finishing";
+ break;
+ case WAIT_EVENT_HASH_GROW_BATCHES_REPARTITIONING:
+ event_name = "Hash/GrowBatches/Repartitioning";
+ break;
+ case WAIT_EVENT_HASH_GROW_BUCKETS_ALLOCATING:
+ event_name = "Hash/GrowBuckets/Allocating";
+ break;
+ case WAIT_EVENT_HASH_GROW_BUCKETS_ELECTING:
+ event_name = "Hash/GrowBuckets/Electing";
+ break;
+ case WAIT_EVENT_HASH_GROW_BUCKETS_REINSERTING:
+ event_name = "Hash/GrowBuckets/Reinserting";
+ break;
+ case WAIT_EVENT_LOGICAL_SYNC_DATA:
+ event_name = "LogicalSyncData";
+ break;
+ case WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE:
+ event_name = "LogicalSyncStateChange";
+ break;
+ case WAIT_EVENT_MQ_INTERNAL:
+ event_name = "MessageQueueInternal";
+ break;
+ case WAIT_EVENT_MQ_PUT_MESSAGE:
+ event_name = "MessageQueuePutMessage";
+ break;
+ case WAIT_EVENT_MQ_RECEIVE:
+ event_name = "MessageQueueReceive";
+ break;
+ case WAIT_EVENT_MQ_SEND:
+ event_name = "MessageQueueSend";
+ break;
+ case WAIT_EVENT_PARALLEL_BITMAP_SCAN:
+ event_name = "ParallelBitmapScan";
+ break;
+ case WAIT_EVENT_PARALLEL_CREATE_INDEX_SCAN:
+ event_name = "ParallelCreateIndexScan";
+ break;
+ case WAIT_EVENT_PARALLEL_FINISH:
+ event_name = "ParallelFinish";
+ break;
+ case WAIT_EVENT_PROCARRAY_GROUP_UPDATE:
+ event_name = "ProcArrayGroupUpdate";
+ break;
+ case WAIT_EVENT_PROMOTE:
+ event_name = "Promote";
+ break;
+ case WAIT_EVENT_REPLICATION_ORIGIN_DROP:
+ event_name = "ReplicationOriginDrop";
+ break;
+ case WAIT_EVENT_REPLICATION_SLOT_DROP:
+ event_name = "ReplicationSlotDrop";
+ break;
+ case WAIT_EVENT_SAFE_SNAPSHOT:
+ event_name = "SafeSnapshot";
+ break;
+ case WAIT_EVENT_SYNC_REP:
+ event_name = "SyncRep";
+ break;
+ /* no default case, so that compiler will warn */
+ }
+
+ return event_name;
+}
+
+/* ----------
+ * pgstat_get_wait_timeout() -
+ *
+ * Convert WaitEventTimeout to string.
+ * ----------
+ */
+static const char *
+pgstat_get_wait_timeout(WaitEventTimeout w)
+{
+ const char *event_name = "unknown wait event";
+
+ switch (w)
+ {
+ case WAIT_EVENT_BASE_BACKUP_THROTTLE:
+ event_name = "BaseBackupThrottle";
+ break;
+ case WAIT_EVENT_PG_SLEEP:
+ event_name = "PgSleep";
+ break;
+ case WAIT_EVENT_RECOVERY_APPLY_DELAY:
+ event_name = "RecoveryApplyDelay";
+ break;
+ /* no default case, so that compiler will warn */
+ }
+
+ return event_name;
+}
+
+/* ----------
+ * pgstat_get_wait_io() -
+ *
+ * Convert WaitEventIO to string.
+ * ----------
+ */
+static const char *
+pgstat_get_wait_io(WaitEventIO w)
+{
+ const char *event_name = "unknown wait event";
+
+ switch (w)
+ {
+ case WAIT_EVENT_BUFFILE_READ:
+ event_name = "BufFileRead";
+ break;
+ case WAIT_EVENT_BUFFILE_WRITE:
+ event_name = "BufFileWrite";
+ break;
+ case WAIT_EVENT_CONTROL_FILE_READ:
+ event_name = "ControlFileRead";
+ break;
+ case WAIT_EVENT_CONTROL_FILE_SYNC:
+ event_name = "ControlFileSync";
+ break;
+ case WAIT_EVENT_CONTROL_FILE_SYNC_UPDATE:
+ event_name = "ControlFileSyncUpdate";
+ break;
+ case WAIT_EVENT_CONTROL_FILE_WRITE:
+ event_name = "ControlFileWrite";
+ break;
+ case WAIT_EVENT_CONTROL_FILE_WRITE_UPDATE:
+ event_name = "ControlFileWriteUpdate";
+ break;
+ case WAIT_EVENT_COPY_FILE_READ:
+ event_name = "CopyFileRead";
+ break;
+ case WAIT_EVENT_COPY_FILE_WRITE:
+ event_name = "CopyFileWrite";
+ break;
+ case WAIT_EVENT_DATA_FILE_EXTEND:
+ event_name = "DataFileExtend";
+ break;
+ case WAIT_EVENT_DATA_FILE_FLUSH:
+ event_name = "DataFileFlush";
+ break;
+ case WAIT_EVENT_DATA_FILE_IMMEDIATE_SYNC:
+ event_name = "DataFileImmediateSync";
+ break;
+ case WAIT_EVENT_DATA_FILE_PREFETCH:
+ event_name = "DataFilePrefetch";
+ break;
+ case WAIT_EVENT_DATA_FILE_READ:
+ event_name = "DataFileRead";
+ break;
+ case WAIT_EVENT_DATA_FILE_SYNC:
+ event_name = "DataFileSync";
+ break;
+ case WAIT_EVENT_DATA_FILE_TRUNCATE:
+ event_name = "DataFileTruncate";
+ break;
+ case WAIT_EVENT_DATA_FILE_WRITE:
+ event_name = "DataFileWrite";
+ break;
+ case WAIT_EVENT_DSM_FILL_ZERO_WRITE:
+ event_name = "DSMFillZeroWrite";
+ break;
+ case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ:
+ event_name = "LockFileAddToDataDirRead";
+ break;
+ case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC:
+ event_name = "LockFileAddToDataDirSync";
+ break;
+ case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE:
+ event_name = "LockFileAddToDataDirWrite";
+ break;
+ case WAIT_EVENT_LOCK_FILE_CREATE_READ:
+ event_name = "LockFileCreateRead";
+ break;
+ case WAIT_EVENT_LOCK_FILE_CREATE_SYNC:
+ event_name = "LockFileCreateSync";
+ break;
+ case WAIT_EVENT_LOCK_FILE_CREATE_WRITE:
+ event_name = "LockFileCreateWrite";
+ break;
+ case WAIT_EVENT_LOCK_FILE_RECHECKDATADIR_READ:
+ event_name = "LockFileReCheckDataDirRead";
+ break;
+ case WAIT_EVENT_LOGICAL_REWRITE_CHECKPOINT_SYNC:
+ event_name = "LogicalRewriteCheckpointSync";
+ break;
+ case WAIT_EVENT_LOGICAL_REWRITE_MAPPING_SYNC:
+ event_name = "LogicalRewriteMappingSync";
+ break;
+ case WAIT_EVENT_LOGICAL_REWRITE_MAPPING_WRITE:
+ event_name = "LogicalRewriteMappingWrite";
+ break;
+ case WAIT_EVENT_LOGICAL_REWRITE_SYNC:
+ event_name = "LogicalRewriteSync";
+ break;
+ case WAIT_EVENT_LOGICAL_REWRITE_TRUNCATE:
+ event_name = "LogicalRewriteTruncate";
+ break;
+ case WAIT_EVENT_LOGICAL_REWRITE_WRITE:
+ event_name = "LogicalRewriteWrite";
+ break;
+ case WAIT_EVENT_RELATION_MAP_READ:
+ event_name = "RelationMapRead";
+ break;
+ case WAIT_EVENT_RELATION_MAP_SYNC:
+ event_name = "RelationMapSync";
+ break;
+ case WAIT_EVENT_RELATION_MAP_WRITE:
+ event_name = "RelationMapWrite";
+ break;
+ case WAIT_EVENT_REORDER_BUFFER_READ:
+ event_name = "ReorderBufferRead";
+ break;
+ case WAIT_EVENT_REORDER_BUFFER_WRITE:
+ event_name = "ReorderBufferWrite";
+ break;
+ case WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ:
+ event_name = "ReorderLogicalMappingRead";
+ break;
+ case WAIT_EVENT_REPLICATION_SLOT_READ:
+ event_name = "ReplicationSlotRead";
+ break;
+ case WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC:
+ event_name = "ReplicationSlotRestoreSync";
+ break;
+ case WAIT_EVENT_REPLICATION_SLOT_SYNC:
+ event_name = "ReplicationSlotSync";
+ break;
+ case WAIT_EVENT_REPLICATION_SLOT_WRITE:
+ event_name = "ReplicationSlotWrite";
+ break;
+ case WAIT_EVENT_SLRU_FLUSH_SYNC:
+ event_name = "SLRUFlushSync";
+ break;
+ case WAIT_EVENT_SLRU_READ:
+ event_name = "SLRURead";
+ break;
+ case WAIT_EVENT_SLRU_SYNC:
+ event_name = "SLRUSync";
+ break;
+ case WAIT_EVENT_SLRU_WRITE:
+ event_name = "SLRUWrite";
+ break;
+ case WAIT_EVENT_SNAPBUILD_READ:
+ event_name = "SnapbuildRead";
+ break;
+ case WAIT_EVENT_SNAPBUILD_SYNC:
+ event_name = "SnapbuildSync";
+ break;
+ case WAIT_EVENT_SNAPBUILD_WRITE:
+ event_name = "SnapbuildWrite";
+ break;
+ case WAIT_EVENT_TIMELINE_HISTORY_FILE_SYNC:
+ event_name = "TimelineHistoryFileSync";
+ break;
+ case WAIT_EVENT_TIMELINE_HISTORY_FILE_WRITE:
+ event_name = "TimelineHistoryFileWrite";
+ break;
+ case WAIT_EVENT_TIMELINE_HISTORY_READ:
+ event_name = "TimelineHistoryRead";
+ break;
+ case WAIT_EVENT_TIMELINE_HISTORY_SYNC:
+ event_name = "TimelineHistorySync";
+ break;
+ case WAIT_EVENT_TIMELINE_HISTORY_WRITE:
+ event_name = "TimelineHistoryWrite";
+ break;
+ case WAIT_EVENT_TWOPHASE_FILE_READ:
+ event_name = "TwophaseFileRead";
+ break;
+ case WAIT_EVENT_TWOPHASE_FILE_SYNC:
+ event_name = "TwophaseFileSync";
+ break;
+ case WAIT_EVENT_TWOPHASE_FILE_WRITE:
+ event_name = "TwophaseFileWrite";
+ break;
+ case WAIT_EVENT_WALSENDER_TIMELINE_HISTORY_READ:
+ event_name = "WALSenderTimelineHistoryRead";
+ break;
+ case WAIT_EVENT_WAL_BOOTSTRAP_SYNC:
+ event_name = "WALBootstrapSync";
+ break;
+ case WAIT_EVENT_WAL_BOOTSTRAP_WRITE:
+ event_name = "WALBootstrapWrite";
+ break;
+ case WAIT_EVENT_WAL_COPY_READ:
+ event_name = "WALCopyRead";
+ break;
+ case WAIT_EVENT_WAL_COPY_SYNC:
+ event_name = "WALCopySync";
+ break;
+ case WAIT_EVENT_WAL_COPY_WRITE:
+ event_name = "WALCopyWrite";
+ break;
+ case WAIT_EVENT_WAL_INIT_SYNC:
+ event_name = "WALInitSync";
+ break;
+ case WAIT_EVENT_WAL_INIT_WRITE:
+ event_name = "WALInitWrite";
+ break;
+ case WAIT_EVENT_WAL_READ:
+ event_name = "WALRead";
+ break;
+ case WAIT_EVENT_WAL_SYNC:
+ event_name = "WALSync";
+ break;
+ case WAIT_EVENT_WAL_SYNC_METHOD_ASSIGN:
+ event_name = "WALSyncMethodAssign";
+ break;
+ case WAIT_EVENT_WAL_WRITE:
+ event_name = "WALWrite";
+ break;
+
+ /* no default case, so that compiler will warn */
+ }
+
+ return event_name;
+}
+
+
+/* ----------
+ * pgstat_get_backend_current_activity() -
+ *
+ * Return a string representing the current activity of the backend with
+ * the specified PID. This looks directly at the BackendStatusArray,
+ * and so will provide current information regardless of the age of our
+ * transaction's snapshot of the status array.
+ *
+ * It is the caller's responsibility to invoke this only for backends whose
+ * state is expected to remain stable while the result is in use. The
+ * only current use is in deadlock reporting, where we can expect that
+ * the target backend is blocked on a lock. (There are corner cases
+ * where the target's wait could get aborted while we are looking at it,
+ * but the very worst consequence is to return a pointer to a string
+ * that's been changed, so we won't worry too much.)
+ *
+ * Note: return strings for special cases match pg_stat_get_backend_activity.
+ * ----------
+ */
+const char *
+pgstat_get_backend_current_activity(int pid, bool checkUser)
+{
+ PgBackendStatus *beentry;
+ int i;
+
+ beentry = BackendStatusArray;
+ for (i = 1; i <= MaxBackends; i++)
+ {
+ /*
+ * Although we expect the target backend's entry to be stable, that
+ * doesn't imply that anyone else's is. To avoid identifying the
+ * wrong backend, while we check for a match to the desired PID we
+ * must follow the protocol of retrying if st_changecount changes
+ * while we examine the entry, or if it's odd. (This might be
+ * unnecessary, since fetching or storing an int is almost certainly
+ * atomic, but let's play it safe.) We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_save_changecount_before(vbeentry, before_changecount);
+
+ found = (vbeentry->st_procpid == pid);
+
+ pgstat_save_changecount_after(vbeentry, after_changecount);
+
+ if (before_changecount == after_changecount &&
+ (before_changecount & 1) == 0)
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ {
+ /* Now it is safe to use the non-volatile pointer */
+ if (checkUser && !superuser() && beentry->st_userid != GetUserId())
+ return "<insufficient privilege>";
+ else if (*(beentry->st_activity_raw) == '\0')
+ return "<command string not enabled>";
+ else
+ {
+ /* this'll leak a bit of memory, but that seems acceptable */
+ return pgstat_clip_activity(beentry->st_activity_raw);
+ }
+ }
+
+ beentry++;
+ }
+
+ /* If we get here, caller is in error ... */
+ return "<backend information not available>";
+}
+
+/* ----------
+ * pgstat_get_crashed_backend_activity() -
+ *
+ * Return a string representing the current activity of the backend with
+ * the specified PID. Like the function above, but reads shared memory with
+ * the expectation that it may be corrupt. On success, copy the string
+ * into the "buffer" argument and return that pointer. On failure,
+ * return NULL.
+ *
+ * This function is only intended to be used by the postmaster to report the
+ * query that crashed a backend. In particular, no attempt is made to
+ * follow the correct concurrency protocol when accessing the
+ * BackendStatusArray. But that's OK, in the worst case we'll return a
+ * corrupted message. We also must take care not to trip on ereport(ERROR).
+ * ----------
+ */
+const char *
+pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
+{
+ volatile PgBackendStatus *beentry;
+ int i;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return NULL;
+
+ for (i = 1; i <= MaxBackends; i++)
+ {
+ if (beentry->st_procpid == pid)
+ {
+ /* Read pointer just once, so it can't change after validation */
+ const char *activity = beentry->st_activity_raw;
+ const char *activity_last;
+
+ /*
+ * We mustn't access activity string before we verify that it
+ * falls within the BackendActivityBuffer. To make sure that the
+ * entire string including its ending is contained within the
+ * buffer, subtract one activity length from the buffer size.
+ */
+ activity_last = BackendActivityBuffer + BackendActivityBufferSize
+ - pgstat_track_activity_query_size;
+
+ if (activity < BackendActivityBuffer ||
+ activity > activity_last)
+ return NULL;
+
+ /* If no string available, no point in a report */
+ if (activity[0] == '\0')
+ return NULL;
+
+ /*
+ * Copy only ASCII-safe characters so we don't run into encoding
+ * problems when reporting the message; and be sure not to run off
+ * the end of memory. As only ASCII characters are reported, it
+ * doesn't seem necessary to perform multibyte aware clipping.
+ */
+ ascii_safe_strlcpy(buffer, activity,
+ Min(buflen, pgstat_track_activity_query_size));
+
+ return buffer;
+ }
+
+ beentry++;
+ }
+
+ /* PID not found */
+ return NULL;
+}
+
+const char *
+pgstat_get_backend_desc(BackendType backendType)
+{
+ const char *backendDesc = "unknown process type";
+
+ switch (backendType)
+ {
+ case B_AUTOVAC_LAUNCHER:
+ backendDesc = "autovacuum launcher";
+ break;
+ case B_AUTOVAC_WORKER:
+ backendDesc = "autovacuum worker";
+ break;
+ case B_BACKEND:
+ backendDesc = "client backend";
+ break;
+ case B_BG_WORKER:
+ backendDesc = "background worker";
+ break;
+ case B_BG_WRITER:
+ backendDesc = "background writer";
+ break;
+ case B_ARCHIVER:
+ backendDesc = "archiver";
+ break;
+ case B_CHECKPOINTER:
+ backendDesc = "checkpointer";
+ break;
+ case B_STARTUP:
+ backendDesc = "startup";
+ break;
+ case B_WAL_RECEIVER:
+ backendDesc = "walreceiver";
+ break;
+ case B_WAL_SENDER:
+ backendDesc = "walsender";
+ break;
+ case B_WAL_WRITER:
+ backendDesc = "walwriter";
+ break;
+ }
+
+ return backendDesc;
+}
+
+/* ----------
+ * pgstat_report_appname() -
+ *
+ * Called to update our application name.
+ * ----------
+ */
+void
+pgstat_report_appname(const char *appname)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+ int len;
+
+ if (!beentry)
+ return;
+
+ /* This should be unnecessary if GUC did its job, but be safe */
+ len = pg_mbcliplen(appname, strlen(appname), NAMEDATALEN - 1);
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ pgstat_increment_changecount_before(beentry);
+
+ memcpy((char *) beentry->st_appname, appname, len);
+ beentry->st_appname[len] = '\0';
+
+ pgstat_increment_changecount_after(beentry);
+}
+
+/*
+ * Report current transaction start timestamp as the specified value.
+ * Zero means there is no active transaction.
+ */
+void
+pgstat_report_xact_timestamp(TimestampTz tstamp)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!pgstat_track_activities || !beentry)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ pgstat_increment_changecount_before(beentry);
+ beentry->st_xact_start_timestamp = tstamp;
+ pgstat_increment_changecount_after(beentry);
+}
+
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgBeStatLocalContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgBeStatLocalContext)
+ pgBeStatLocalContext = AllocSetContextCreate(TopMemoryContext,
+ "Backend status snapshot",
+ ALLOCSET_SMALL_SIZES);
+}
+
+/* ----------
+ * AtEOXact_BEStatus
+ *
+ * Called from access/transam/xact.c at top-level transaction commit/abort.
+ * ----------
+ */
+void
+AtEOXact_BEStatus(bool isCommit)
+{
+ bestatus_clear_snapshot();
+}
+
+/*
+ * AtPrepare_BEStatus
+ * Clear existing snapshot at 2PC transaction prepare.
+ */
+void
+AtPrepare_BEStatus(void)
+{
+ bestatus_clear_snapshot();
+}
+
+/* ----------
+ * bestatus_clear_snapshot() -
+ *
+ * Discard any data collected in the current transaction. Any subsequent
+ * request will cause new snapshots to be read.
+ *
+ * This is also invoked during transaction commit or abort to discard
+ * the no-longer-wanted snapshot.
+ * ----------
+ */
+static void
+bestatus_clear_snapshot(void)
+{
+ /* Release memory, if any was allocated */
+ if (pgBeStatLocalContext)
+ MemoryContextDelete(pgBeStatLocalContext);
+
+ /* Reset variables */
+ pgBeStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
+}
+
+
+
+/* ----------
+ * pgstat_report_activity() -
+ *
+ * Called from tcop/postgres.c to report what the backend is actually doing
+ * (but note cmd_str can be NULL for certain cases).
+ *
+ * All updates of the status entry follow the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ * ----------
+ */
+void
+pgstat_report_activity(BackendState state, const char *cmd_str)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+ TimestampTz start_timestamp;
+ TimestampTz current_timestamp;
+ int len = 0;
+
+ TRACE_POSTGRESQL_STATEMENT_STATUS(cmd_str);
+
+ if (!beentry)
+ return;
+
+ if (!pgstat_track_activities)
+ {
+ if (beentry->st_state != STATE_DISABLED)
+ {
+ volatile PGPROC *proc = MyProc;
+
+ /*
+ * track_activities is disabled, but we last reported a
+ * non-disabled state. As our final update, change the state and
+ * clear fields we will not be updating anymore.
+ */
+ pgstat_increment_changecount_before(beentry);
+ beentry->st_state = STATE_DISABLED;
+ beentry->st_state_start_timestamp = 0;
+ beentry->st_activity_raw[0] = '\0';
+ beentry->st_activity_start_timestamp = 0;
+ /* st_xact_start_timestamp and wait_event_info are also disabled */
+ beentry->st_xact_start_timestamp = 0;
+ proc->wait_event_info = 0;
+ pgstat_increment_changecount_after(beentry);
+ }
+ return;
+ }
+
+ /*
+ * To minimize the time spent modifying the entry, fetch all the needed
+ * data first.
+ */
+ start_timestamp = GetCurrentStatementStartTimestamp();
+ if (cmd_str != NULL)
+ {
+ /*
+ * Compute length of to-be-stored string unaware of multi-byte
+ * characters. For speed reasons that'll get corrected on read, rather
+ * than computed every write.
+ */
+ len = Min(strlen(cmd_str), pgstat_track_activity_query_size - 1);
+ }
+ current_timestamp = GetCurrentTimestamp();
+
+ /*
+ * Now update the status entry
+ */
+ pgstat_increment_changecount_before(beentry);
+
+ beentry->st_state = state;
+ beentry->st_state_start_timestamp = current_timestamp;
+
+ if (cmd_str != NULL)
+ {
+ memcpy((char *) beentry->st_activity_raw, cmd_str, len);
+ beentry->st_activity_raw[len] = '\0';
+ beentry->st_activity_start_timestamp = start_timestamp;
+ }
+
+ pgstat_increment_changecount_after(beentry);
+}
+
+/*-----------
+ * pgstat_progress_start_command() -
+ *
+ * Set st_progress_command (and st_progress_command_target) in own backend
+ * entry. Also, zero-initialize st_progress_param array.
+ *-----------
+ */
+void
+pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ pgstat_increment_changecount_before(beentry);
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ pgstat_increment_changecount_after(beentry);
+}
+
+/*-----------
+ * pgstat_progress_update_param() -
+ *
+ * Update index'th member in st_progress_param[] of own backend entry.
+ *-----------
+ */
+void
+pgstat_progress_update_param(int index, int64 val)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ pgstat_increment_changecount_before(beentry);
+ beentry->st_progress_param[index] = val;
+ pgstat_increment_changecount_after(beentry);
+}
+
+/*-----------
+ * pgstat_progress_update_multi_param() -
+ *
+ * Update multiple members in st_progress_param[] of own backend entry.
+ * This is atomic; readers won't see intermediate states.
+ *-----------
+ */
+void
+pgstat_progress_update_multi_param(int nparam, const int *index,
+ const int64 *val)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+ int i;
+
+ if (!beentry || !pgstat_track_activities || nparam == 0)
+ return;
+
+ pgstat_increment_changecount_before(beentry);
+
+ for (i = 0; i < nparam; ++i)
+ {
+ Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
+
+ beentry->st_progress_param[index[i]] = val[i];
+ }
+
+ pgstat_increment_changecount_after(beentry);
+}
+
+/*-----------
+ * pgstat_progress_end_command() -
+ *
+ * Reset st_progress_command (and st_progress_command_target) in own backend
+ * entry. This signals the end of the command.
+ *-----------
+ */
+void
+pgstat_progress_end_command(void)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry)
+ return;
+ if (!pgstat_track_activities
+ && beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ return;
+
+ pgstat_increment_changecount_before(beentry);
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ pgstat_increment_changecount_after(beentry);
+}
+
+
+/*
+ * Convert a potentially unsafely truncated activity string (see
+ * PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
+ * one.
+ *
+ * The returned string is allocated in the caller's memory context and may be
+ * freed.
+ */
+char *
+pgstat_clip_activity(const char *raw_activity)
+{
+ char *activity;
+ int rawlen;
+ int cliplen;
+
+ /*
+ * Some callers, like pgstat_get_backend_current_activity(), do not
+ * guarantee that the buffer isn't concurrently modified. We try to take
+ * care that the buffer is always terminated by a NUL byte regardless, but
+ * let's still be paranoid about the string's length. In those cases the
+ * underlying buffer is guaranteed to be pgstat_track_activity_query_size
+ * large.
+ */
+ activity = pnstrdup(raw_activity, pgstat_track_activity_query_size - 1);
+
+ /* now double-guaranteed to be NUL terminated */
+ rawlen = strlen(activity);
+
+ /*
+ * All supported server-encodings make it possible to determine the length
+ * of a multi-byte character from its first byte (this is not the case for
+ * client encodings, see GB18030). As st_activity is always stored using
+ * server encoding, this allows us to perform multi-byte aware truncation,
+ * even if the string earlier was truncated in the middle of a multi-byte
+ * character.
+ */
+ cliplen = pg_mbcliplen(activity, rawlen,
+ pgstat_track_activity_query_size - 1);
+
+ activity[cliplen] = '\0';
+
+ return activity;
+}
diff --git a/src/backend/statmon/pgstat.c b/src/backend/statmon/pgstat.c
new file mode 100644
index 0000000000..66fb1e9341
--- /dev/null
+++ b/src/backend/statmon/pgstat.c
@@ -0,0 +1,3962 @@
+/* ----------
+ * pgstat.c
+ *
+ * Statistics collector facility.
+ *
+ * Collects per-table and per-function usage statitics of backends and shares
+ * them among all backends via shared memory. Every backend records
+ * individual activity in local memory using pg_count_*() and friends
+ * interfaces during a transaction. Then pgstat_report_stat() is called at
+ * the end of a transacion to flush out the local numbers to shared
+ * memory. To avoid congestion on the shared memory, we do that not often
+ * than PGSTAT_STAT_MIN_INTERVAL(500ms). Still it is possible that a backend
+ * cannot flush all or a part of local numbers immediately, such numbers are
+ * postponed to the next chance but they are not kept longer than
+ * PGSTAT_STAT_MAX_INTERVAL(1000ms).
+ *
+ * pgstat_fetch_stat_*() are used to read the statistics numbers. There are
+ * two ways of reading the shared statistics. Transactional and
+ * one-shot. Retrieved numbers are stored in local hash which persists until
+ * transaction-end in the former type. One the other hand autovacuum, which
+ * doesn't need such characteristics, uses one-shot mode, which just copies
+ * the data into palloc'ed memory.
+ *
+ * Copyright (c) 2001-2019, PostgreSQL Global Development Group
+ *
+ * src/backend/statmon/pgstat.c
+ * ----------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "pgstat.h"
+
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/twophase_rmgr.h"
+#include "access/xact.h"
+#include "bestatus.h"
+#include "catalog/pg_database.h"
+#include "catalog/pg_proc.h"
+#include "miscadmin.h"
+#include "postmaster/autovacuum.h"
+#include "storage/ipc.h"
+#include "storage/lmgr.h"
+#include "storage/procsignal.h"
+#include "utils/memutils.h"
+#include "utils/snapmgr.h"
+
+/* ----------
+ * Timer definitions.
+ * ----------
+ */
+#define PGSTAT_STAT_MIN_INTERVAL 500 /* Minimum time between stats data
+ * updates; in milliseconds. */
+
+#define PGSTAT_STAT_MAX_INTERVAL 1000 /* Maximum time between stats data
+ * updates; in milliseconds. */
+
+/* ----------
+ * The initial size hints for the hash tables used in the collector.
+ * ----------
+ */
+#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_FUNCTION_HASH_SIZE 512
+
+/*
+ * Operation mode of pgstat_get_db_entry.
+ */
+#define PGSTAT_FETCH_SHARED 0
+#define PGSTAT_FETCH_EXCLUSIVE 1
+#define PGSTAT_FETCH_NOWAIT 2
+
+typedef enum PgStat_TableLookupState;
+{
+ PGSTAT_ENTRY_NOT_FOUND,
+ PGSTAT_ENTRY_FOUND,
+ PGSTAT_ENTRY_LOCK_FAILED
+} PgStat_TableLookupState;
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+bool pgstat_track_counts = false;
+int pgstat_track_functions = TRACK_FUNC_OFF;
+
+/* ----------
+ * Built from GUC parameter
+ * ----------
+ */
+char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
+char *pgstat_stat_filename = NULL;
+char *pgstat_stat_tmpname = NULL;
+
+/* Shared stats bootstrap infomation */
+typedef struct StatsShmemStruct {
+ dsa_handle stats_dsa_handle;
+ dshash_table_handle db_stats_handle;
+ dsa_pointer global_stats;
+ dsa_pointer archiver_stats;
+ TimestampTz last_update;
+} StatsShmemStruct;
+
+
+/*
+ * BgWriter global statistics counters (unused in other processes).
+ * Stored directly in a stats message structure so it can be sent
+ * without needing to copy things around. We assume this inits to zeroes.
+ */
+PgStat_BgWriter BgWriterStats;
+
+/* ----------
+ * Local data
+ * ----------
+ */
+/* Variables lives for the backend lifetime */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
+static dshash_table *db_stats;
+
+/* memory context for snapshots */
+static MemoryContext pgStatLocalContext = NULL;
+static HTAB *snapshot_db_stats;
+
+/* dshash parameter for each type of table */
+static const dshash_parameters dsh_dbparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatDBEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS_DB
+};
+static const dshash_parameters dsh_tblparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatTabEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS_FUNC_TABLE
+};
+static const dshash_parameters dsh_funcparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatFuncEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS_FUNC_TABLE
+};
+
+/*
+ * Structures in which backends store per-table info that's waiting to be
+ * written to shared stats.
+ *
+ * NOTE: once allocated, TabStatusArray structures are never moved or deleted
+ * for the life of the backend. Also, we zero out the t_id fields of the
+ * contained PgStat_TableStatus structs whenever they are not actively in use.
+ * This allows relcache pgstat_info pointers to be treated as long-lived data,
+ * avoiding repeated searches in pgstat_initstats() when a relation is
+ * repeatedly opened during a transaction.
+ */
+#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
+
+typedef struct TabStatusArray
+{
+ struct TabStatusArray *tsa_next; /* link to next array, if any */
+ int tsa_used; /* # entries currently used */
+ PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
+} TabStatusArray;
+
+static TabStatusArray *pgStatTabList = NULL;
+
+/*
+ * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ */
+typedef struct TabStatHashEntry
+{
+ Oid t_id;
+ PgStat_TableStatus *tsa_entry;
+} TabStatHashEntry;
+
+/*
+ * Hash table for O(1) t_id -> tsa_entry lookup
+ */
+static HTAB *pgStatTabHash = NULL;
+
+/*
+ * Backends store per-function info that's waiting to be flushed out to shared
+ * memory in this hash table (indexed by function OID).
+ */
+static HTAB *pgStatFunctions = NULL;
+
+/*
+ * variables signal that the backend has numbers that have not been able to
+ * be flushed out to shared memory in the past trials.
+ */
+static bool pgStatPendingRecoveryConflicts = false;
+static bool pgStatPendingDeadlocks = false;
+static bool pgStatPendingTempfiles = false;
+static HTAB *pgStatPendingTabHash = NULL;
+static HTAB *pgStatPendingFunctions = NULL;
+
+/*
+ * Tuple insertion/deletion counts for an open transaction can't be propagated
+ * into PgStat_TableStatus counters until we know if it is going to commit
+ * or abort. Hence, we keep these counts in per-subxact structs that live
+ * in TopTransactionContext. This data structure is designed on the assumption
+ * that subxacts won't usually modify very many tables.
+ */
+typedef struct PgStat_SubXactStatus
+{
+ int nest_level; /* subtransaction nest level */
+ struct PgStat_SubXactStatus *prev; /* higher-level subxact if any */
+ PgStat_TableXactStatus *first; /* head of list for this subxact */
+} PgStat_SubXactStatus;
+
+static PgStat_SubXactStatus *pgStatXactStack = NULL;
+
+static int pgStatXactCommit = 0;
+static int pgStatXactRollback = 0;
+PgStat_Counter pgStatBlockReadTime = 0;
+PgStat_Counter pgStatBlockWriteTime = 0;
+
+/* Record that's written to 2PC state file when pgstat state is persisted */
+typedef struct TwoPhasePgStatRecord
+{
+ PgStat_Counter tuples_inserted; /* tuples inserted in xact */
+ PgStat_Counter tuples_updated; /* tuples updated in xact */
+ PgStat_Counter tuples_deleted; /* tuples deleted in xact */
+ PgStat_Counter inserted_pre_trunc; /* tuples inserted prior to truncate */
+ PgStat_Counter updated_pre_trunc; /* tuples updated prior to truncate */
+ PgStat_Counter deleted_pre_trunc; /* tuples deleted prior to truncate */
+ Oid t_id; /* table's OID */
+ bool t_shared; /* is it a shared catalog? */
+ bool t_truncated; /* was the relation truncated? */
+} TwoPhasePgStatRecord;
+
+typedef struct
+{
+ dshash_table *tabhash;
+ PgStat_StatDBEntry *dbentry;
+} pgstat_apply_tabstat_context;
+
+/*
+ * Info about current snapshot of stats
+ */
+TimestampTz backend_cache_expire = 0; /* local cache expiration time */
+bool first_in_xact = true; /* first fetch after the last tr end */
+
+/*
+ * Cluster wide statistics.
+
+ * Contains statistics that are not collected per database or per table.
+ * shared_* are the statistics maintained by statistics collector code and
+ * snapshot_* are cached stats for the reader code.
+ */
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats *snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats *snapshot_globalStats;
+
+/*
+ * Total time charged to functions so far in the current backend.
+ * We use this to help separate "self" and "other" time charges.
+ * (We assume this initializes to zero.)
+ */
+static instr_time total_func_time;
+
+
+/* ----------
+ * Local function forward declarations
+ * ----------
+ */
+/* functions used in backends */
+static void pgstat_beshutdown_hook(int code, Datum arg);
+
+static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, int op,
+ PgStat_TableLookupState *status);
+static PgStat_StatTabEntry *pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create);
+static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_db_statsfile(Oid databaseid, dshash_table *tabhash, dshash_table *funchash);
+
+/* functions used in backends */
+static bool backend_snapshot_global_stats(void);
+static PgStat_StatFuncEntry *backend_get_func_etnry(PgStat_StatDBEntry *dbent, Oid funcid, bool oneshot);
+
+static void pgstat_postmaster_shutdown(int code, Datum arg);
+static void pgstat_apply_pending_tabstats(bool shared, bool force,
+ pgstat_apply_tabstat_context *cxt);
+static bool pgstat_apply_tabstat(pgstat_apply_tabstat_context *cxt,
+ PgStat_TableStatus *entry, bool nowait);
+static void pgstat_merge_tabentry(PgStat_TableStatus *deststat,
+ PgStat_TableStatus *srcstat,
+ bool init);
+static void pgstat_update_funcstats(bool force, PgStat_StatDBEntry *dbentry);
+static void pgstat_reset_all_counters(void);
+static void pgstat_cleanup_recovery_conflict(PgStat_StatDBEntry *dbentry);
+static void pgstat_cleanup_deadlock(PgStat_StatDBEntry *dbentry);
+static void pgstat_cleanup_tempfile(PgStat_StatDBEntry *dbentry);
+
+static inline void pgstat_merge_backendstats_to_funcentry(
+ PgStat_StatFuncEntry *dest, PgStat_BackendFunctionEntry *src, bool init);
+static inline void pgstat_merge_funcentry(
+ PgStat_StatFuncEntry *dest, PgStat_StatFuncEntry *src, bool init);
+
+static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
+static void reset_dbentry_counters(PgStat_StatDBEntry *dbentry);
+static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
+
+static void pgstat_setup_memcxt(void);
+
+static bool pgstat_update_tabentry(dshash_table *tabhash,
+ PgStat_TableStatus *stat, bool nowait);
+static void pgstat_update_dbentry(PgStat_StatDBEntry *dbentry,
+ PgStat_TableStatus *stat);
+
+/* ------------------------------------------------------------
+ * Public functions called from postmaster follow
+ * ------------------------------------------------------------
+ */
+
+
+void
+pgstat_initialize(void)
+{
+ /* Set up a process-exit hook to clean up */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
+}
+
+/*
+ * subroutine for pgstat_reset_all
+ */
+static void
+pgstat_reset_remove_files(const char *directory)
+{
+ DIR *dir;
+ struct dirent *entry;
+ char fname[MAXPGPATH * 2];
+
+ dir = AllocateDir(directory);
+ while ((entry = ReadDir(dir, directory)) != NULL)
+ {
+ int nchars;
+ Oid tmp_oid;
+
+ /*
+ * Skip directory entries that don't match the file names we write.
+ * See get_dbstat_filename for the database-specific pattern.
+ */
+ if (strncmp(entry->d_name, "global.", 7) == 0)
+ nchars = 7;
+ else
+ {
+ nchars = 0;
+ (void) sscanf(entry->d_name, "db_%u.%n",
+ &tmp_oid, &nchars);
+ if (nchars <= 0)
+ continue;
+ /* %u allows leading whitespace, so reject that */
+ if (strchr("0123456789", entry->d_name[3]) == NULL)
+ continue;
+ }
+
+ if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
+ strcmp(entry->d_name + nchars, "stat") != 0)
+ continue;
+
+ snprintf(fname, sizeof(fname), "%s/%s", directory,
+ entry->d_name);
+ unlink(fname);
+ }
+ FreeDir(dir);
+}
+
+/*
+ * pgstat_reset_all() -
+ *
+ * Remove the stats files and on-memory counters. This is currently used only
+ * if WAL recovery is needed after a crash.
+ */
+void
+pgstat_reset_all(void)
+{
+ pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
+ pgstat_reset_all_counters();
+}
+
+/* ----------
+ * pgstat_create_shared_stats() -
+ *
+ * create shared stats memory
+ * ----------
+ */
+static void
+pgstat_create_shared_stats(void)
+{
+ MemoryContext oldcontext;
+
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
+
+ /* lives for the lifetime of the process */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ area = dsa_create(LWTRANCHE_STATS_DSA);
+ dsa_pin_mapping(area);
+
+ db_stats = dshash_create(area, &dsh_dbparams, 0);
+
+ /* create shared area and write bootstrap information */
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->db_stats_handle =
+ dshash_get_hash_table_handle(db_stats);
+ StatsShmem->last_update = 0;
+
+ /* initial connect to the memory */
+ MemoryContextSwitchTo(pgStatLocalContext);
+ snapshot_db_stats = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
+ MemoryContextSwitchTo(oldcontext);
+}
+
+
+/* ------------------------------------------------------------
+ * Public functions used by backends follow
+ *------------------------------------------------------------
+ */
+
+
+/* ----------
+ * pgstat_report_stat() -
+ *
+ * Must be called by processes that performs DML: tcop/postgres.c, logical
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * This requires taking some locks on the shared statistics hashes and some
+ * of updates may be postponed on lock failure. Such postponed updates are
+ * retried in later call of this function and finally cleaned up by calling
+ * this function with force = true or PGSTAT_STAT_MAX_INTERVAL milliseconds
+ * has elapsed since last cleanup. On the other hand updates by regular
+ * backends happen with the interval not shorter than
+ * PGSTAT_STAT_MIN_INTERVAL when force = false.
+ *
+ * Returns time in milliseconds until the next update time.
+ *
+ * Note that this is called only out of a transaction, so it is fair to use
+ * transaction stop time as an approximation of current time.
+ * ----------
+ */
+long
+pgstat_update_stat(bool force)
+{
+ static TimestampTz last_report = 0;
+ static TimestampTz oldest_pending = 0;
+ TimestampTz now;
+ TabStatusArray *tsa;
+ pgstat_apply_tabstat_context cxt;
+ bool other_pending_stats = false;
+ long elapsed;
+ long secs;
+ int usecs;
+
+ if (pgStatPendingRecoveryConflicts ||
+ pgStatPendingDeadlocks ||
+ pgStatPendingTempfiles ||
+ pgStatPendingFunctions)
+ other_pending_stats = true;
+
+ /* Don't expend a clock check if nothing to do */
+ if (!other_pending_stats && !pgStatPendingTabHash &&
+ (pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
+ pgStatXactCommit == 0 && pgStatXactRollback == 0)
+ return 0;
+
+ now = GetCurrentTransactionStopTimestamp();
+
+ if (!force)
+ {
+ /*
+ * Don't update shared stats unless it's been at least
+ * PGSTAT_STAT_MIN_INTERVAL msec since we last updated one.
+ * Returns time to wait in the case.
+ */
+ TimestampDifference(last_report, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs /1000;
+
+ if(elapsed < PGSTAT_STAT_MIN_INTERVAL)
+ {
+ /* we know we have some statistics */
+ if (oldest_pending == 0)
+ oldest_pending = now;
+
+ return PGSTAT_STAT_MIN_INTERVAL - elapsed;
+ }
+
+
+ /*
+ * Don't keep pending stats for longer than PGSTAT_STAT_MAX_INTERVAL.
+ */
+ if (oldest_pending > 0)
+ {
+ TimestampDifference(oldest_pending, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs /1000;
+
+ if(elapsed > PGSTAT_STAT_MAX_INTERVAL)
+ force = true;
+ }
+ }
+
+ last_report = now;
+
+ /* Publish report time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (StatsShmem->last_update < last_report)
+ StatsShmem->last_update = last_report;
+ LWLockRelease(StatsLock);
+
+
+ /* setup stats update context*/
+ cxt.dbentry = NULL;
+ cxt.tabhash = NULL;
+
+ /*
+ * Try other pending stats first. Although this may be after flushing
+ * table stats, we do it here to reduce looking up of database entry.
+ */
+ if (other_pending_stats)
+ {
+ int op = PGSTAT_FETCH_EXCLUSIVE;
+
+ if (!force)
+ op |= PGSTAT_FETCH_NOWAIT;
+
+ cxt.dbentry = pgstat_get_db_entry(MyDatabaseId, op, NULL);
+
+ if (cxt.dbentry)
+ {
+ /* clean up pending statistics if any */
+ if (pgStatPendingFunctions)
+ pgstat_update_funcstats(true, cxt.dbentry);
+ if (pgStatPendingRecoveryConflicts)
+ pgstat_cleanup_recovery_conflict(cxt.dbentry);
+ if (pgStatPendingDeadlocks)
+ pgstat_cleanup_deadlock(cxt.dbentry);
+ if (pgStatPendingTempfiles)
+ pgstat_cleanup_tempfile(cxt.dbentry);
+ }
+ }
+
+ /*
+ * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
+ * entries it points to. (Should we fail partway through the loop below,
+ * it's okay to have removed the hashtable already --- the only
+ * consequence is we'd get multiple entries for the same table in the
+ * pgStatTabList, and that's safe.)
+ */
+ if (pgStatTabHash)
+ hash_destroy(pgStatTabHash);
+ pgStatTabHash = NULL;
+
+ /*
+ * Flush pending stats separately for regular tables and shared tables
+ * since we cannot hold locks on two dshash entries at once.
+ */
+
+ /* The first call of the followings uses dbentry obtained above if any.*/
+ pgstat_apply_pending_tabstats(false, force, &cxt);
+ pgstat_apply_pending_tabstats(true, force, &cxt);
+
+ /* zero out TableStatus structs after use */
+ for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ {
+ MemSet(tsa->tsa_entries, 0,
+ tsa->tsa_used * sizeof(PgStat_TableStatus));
+ tsa->tsa_used = 0;
+ }
+
+ /* record oldest pending update time */
+ if (pgStatPendingTabHash == NULL)
+ oldest_pending = 0;
+ else if (oldest_pending == 0)
+ oldest_pending = now;
+
+ return 0;
+}
+
+/*
+ * Subroutine for pgstat_update_stat.
+ *
+ * Applies table stats in table status array merging with pending stats if
+ * any. If force is true waits until required locks to be acquired. Otherwise
+ * stats merged stats as pending stats and it will be processed at the next
+ * chance.
+ */
+static void
+pgstat_apply_pending_tabstats(bool shared, bool force,
+ pgstat_apply_tabstat_context *cxt)
+{
+ static const PgStat_TableCounts all_zeroes;
+ TabStatusArray *tsa;
+ int i;
+
+ for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ {
+ for (i = 0; i < tsa->tsa_used; i++)
+ {
+ PgStat_TableStatus *entry = &tsa->tsa_entries[i];
+ PgStat_TableStatus *pentry = NULL;
+
+ /* Shouldn't have any pending transaction-dependent counts */
+ Assert(entry->trans == NULL);
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such
+ * as indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&entry->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ continue;
+
+ /* Skip if this entry is not match the request */
+ if (entry->t_shared != shared)
+ continue;
+
+ /* if pending update exists, it should be applied along with */
+ if (pgStatPendingTabHash != NULL)
+ {
+ pentry = hash_search(pgStatPendingTabHash,
+ (void *) entry, HASH_FIND, NULL);
+
+ if (pentry)
+ {
+ /* merge new update into pending updates */
+ pgstat_merge_tabentry(pentry, entry, false);
+ entry = pentry;
+ }
+ }
+
+ /* try to apply the merged stats */
+ if (pgstat_apply_tabstat(cxt, entry, !force))
+ {
+ /* succeeded. remove it if it was pending stats */
+ if (pentry)
+ hash_search(pgStatPendingTabHash,
+ (void *) pentry, HASH_REMOVE, NULL);
+ }
+ else if (!pentry)
+ {
+ /* failed and there was no pending entry, create new one. */
+ bool found;
+
+ if (pgStatPendingTabHash == NULL)
+ {
+ HASHCTL ctl;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(PgStat_TableStatus);
+ pgStatPendingTabHash =
+ hash_create("pgstat pending table stats hash",
+ TABSTAT_QUANTUM,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ pentry = hash_search(pgStatPendingTabHash,
+ (void *) entry, HASH_ENTER, &found);
+ Assert (!found);
+
+ *pentry = *entry;
+ }
+ }
+ }
+
+ /* if any pending stats exists, try to clean it up */
+ if (pgStatPendingTabHash != NULL)
+ {
+ HASH_SEQ_STATUS pstat;
+ PgStat_TableStatus *pentry;
+
+ hash_seq_init(&pstat, pgStatPendingTabHash);
+ while((pentry = (PgStat_TableStatus *) hash_seq_search(&pstat)) != NULL)
+ {
+ /* Skip if this entry is not match the request */
+ if (pentry->t_shared != shared)
+ continue;
+
+ /* apply pending entry and remove on success */
+ if (pgstat_apply_tabstat(cxt, pentry, !force))
+ hash_search(pgStatPendingTabHash,
+ (void *) pentry, HASH_REMOVE, NULL);
+ }
+
+ /* destroy the hash if no entry is left */
+ if (hash_get_num_entries(pgStatPendingTabHash) == 0)
+ {
+ hash_destroy(pgStatPendingTabHash);
+ pgStatPendingTabHash = NULL;
+ }
+ }
+
+ if (cxt->tabhash)
+ dshash_detach(cxt->tabhash);
+ if (cxt->dbentry)
+ dshash_release_lock(db_stats, cxt->dbentry);
+ cxt->tabhash = NULL;
+ cxt->dbentry = NULL;
+}
+
+
+/*
+ * pgstat_apply_tabstat: update shared stats entry using given entry
+ *
+ * If nowait is true, just returns false on lock failure. Dshashes for table
+ * and function stats are kept attached and stored in ctx. The caller must
+ * detach them after use.
+ */
+bool
+pgstat_apply_tabstat(pgstat_apply_tabstat_context *cxt,
+ PgStat_TableStatus *entry, bool nowait)
+{
+ Oid dboid = entry->t_shared ? InvalidOid : MyDatabaseId;
+ int table_mode = PGSTAT_FETCH_EXCLUSIVE;
+ bool updated = false;
+
+ if (nowait)
+ table_mode |= PGSTAT_FETCH_NOWAIT;
+
+ /*
+ * We need to keep lock on dbentries for regular tables to avoid race
+ * condition with drop database. So we hold it in the context variable. We
+ * don't need that for shared tables.
+ */
+ if (!cxt->dbentry)
+ cxt->dbentry = pgstat_get_db_entry(dboid, table_mode, NULL);
+
+ /* we cannot acquire lock, just return */
+ if (!cxt->dbentry)
+ return false;
+
+ /* attach shared stats table if not yet */
+ if (!cxt->tabhash)
+ {
+ /*
+ * If we haven't attached the tabhash, we didn't apply database stats
+ * yet. So apply it now..
+ */
+ if (!entry->t_shared)
+ {
+ /* Update database-wide stats */
+ cxt->dbentry->n_xact_commit += pgStatXactCommit;
+ cxt->dbentry->n_xact_rollback += pgStatXactRollback;
+ cxt->dbentry->n_block_read_time += pgStatBlockReadTime;
+ cxt->dbentry->n_block_write_time += pgStatBlockWriteTime;
+ pgStatXactCommit = 0;
+ pgStatXactRollback = 0;
+ pgStatBlockReadTime = 0;
+ pgStatBlockWriteTime = 0;
+ }
+
+ cxt->tabhash =
+ dshash_attach(area, &dsh_tblparams, cxt->dbentry->tables, 0);
+ }
+
+ /*
+ * If we have access to the required data, try update table stats first.
+ * Update database stats only if the first step suceeded.
+ */
+ if (pgstat_update_tabentry(cxt->tabhash, entry, nowait))
+ {
+ pgstat_update_dbentry(cxt->dbentry, entry);
+ updated = true;
+ }
+
+ return updated;
+}
+
+/*
+ * pgstat_merge_tabentry: subroutine for pgstat_update_stat
+ *
+ * Merge srcstat into deststat. Existing value in deststat is cleard if
+ * init is true.
+ */
+static void
+pgstat_merge_tabentry(PgStat_TableStatus *deststat,
+ PgStat_TableStatus *srcstat,
+ bool init)
+{
+ Assert (deststat != srcstat);
+
+ if (init)
+ deststat->t_counts = srcstat->t_counts;
+ else
+ {
+ PgStat_TableCounts *dest = &deststat->t_counts;
+ PgStat_TableCounts *src = &srcstat->t_counts;
+
+ dest->t_numscans += src->t_numscans;
+ dest->t_tuples_returned += src->t_tuples_returned;
+ dest->t_tuples_fetched += src->t_tuples_fetched;
+ dest->t_tuples_inserted += src->t_tuples_inserted;
+ dest->t_tuples_updated += src->t_tuples_updated;
+ dest->t_tuples_deleted += src->t_tuples_deleted;
+ dest->t_tuples_hot_updated += src->t_tuples_hot_updated;
+ dest->t_truncated |= src->t_truncated;
+
+ /* If table was truncated, first reset the live/dead counters */
+ if (src->t_truncated)
+ {
+ dest->t_delta_live_tuples = 0;
+ dest->t_delta_dead_tuples = 0;
+ }
+ dest->t_delta_live_tuples += src->t_delta_live_tuples;
+ dest->t_delta_dead_tuples += src->t_delta_dead_tuples;
+ dest->t_changed_tuples += src->t_changed_tuples;
+ dest->t_blocks_fetched += src->t_blocks_fetched;
+ dest->t_blocks_hit += src->t_blocks_hit;
+ }
+}
+
+/*
+ * pgstat_update_funcstats: subroutine for pgstat_update_stat
+ *
+ * updates a function stat
+ */
+static void
+pgstat_update_funcstats(bool force, PgStat_StatDBEntry *dbentry)
+{
+ /* we assume this inits to all zeroes: */
+ static const PgStat_FunctionCounts all_zeroes;
+ PgStat_TableLookupState status = 0;
+ dshash_table *funchash;
+ bool nowait = !force;
+ bool release_db = false;
+ int table_op = PGSTAT_FETCH_EXCLUSIVE;
+
+ if (pgStatFunctions == NULL && pgStatPendingFunctions == NULL)
+ return;
+
+ if (nowait)
+ table_op += PGSTAT_FETCH_NOWAIT;
+
+ /* find the shared function stats table */
+ if (!dbentry)
+ {
+ dbentry = pgstat_get_db_entry(MyDatabaseId, table_op, &status);
+ release_db = true;
+ }
+
+ /* lock failure, return. */
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ /*
+ * create hash if not yet, we don't keep shared memory dead when function
+ * stats are not being taken.
+ */
+ if (dbentry->functions == DSM_HANDLE_INVALID)
+ {
+ funchash = dshash_create(area, &dsh_funcparams, 0);
+ dbentry->functions = dshash_get_hash_table_handle(funchash);
+ }
+ else
+ funchash = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+
+ /*
+ * First, we empty the transaction stats. Just move numbers to pending
+ * stats if any. Elsewise try to directly update the shared stats but
+ * create a new pending entry on lock failure.
+ */
+ if (pgStatFunctions)
+ {
+ HASH_SEQ_STATUS fstat;
+ PgStat_BackendFunctionEntry *bestat;
+
+ hash_seq_init(&fstat, pgStatFunctions);
+ while ((bestat = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ {
+ bool found;
+ bool init = false;
+ PgStat_StatFuncEntry *funcent = NULL;
+
+ /* Skip it if no counts accumulated since last time */
+ if (memcmp(&bestat->f_counts, &all_zeroes,
+ sizeof(PgStat_FunctionCounts)) == 0)
+ continue;
+
+ /* find pending entry */
+ if (pgStatPendingFunctions)
+ funcent = (PgStat_StatFuncEntry *)
+ hash_search(pgStatPendingFunctions,
+ (void *) &(bestat->f_id), HASH_FIND, NULL);
+
+ if (!funcent)
+ {
+ /* pending entry not found, find shared stats entry */
+ funcent = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert_extended(funchash,
+ (void *) &(bestat->f_id),
+ &found, nowait);
+ if (funcent)
+ init = !found;
+ else
+ {
+ /* no shared stats entry. create a new pending one */
+ funcent = (PgStat_StatFuncEntry *)
+ hash_search(pgStatPendingFunctions,
+ (void *) &(bestat->f_id), HASH_ENTER, NULL);
+ init = true;
+ }
+ }
+ Assert (funcent != NULL);
+
+ pgstat_merge_backendstats_to_funcentry(funcent, bestat, init);
+
+ /* reset used counts */
+ MemSet(&bestat->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ }
+ }
+
+ /* Second, apply pending stats numbers to shared table */
+ if (pgStatPendingFunctions)
+ {
+ HASH_SEQ_STATUS fstat;
+ PgStat_StatFuncEntry *pendent;
+
+ hash_seq_init(&fstat, pgStatPendingFunctions);
+ while ((pendent = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ {
+ PgStat_StatFuncEntry *funcent;
+ bool found;
+
+ funcent = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert_extended(funchash,
+ (void *) &(pendent->functionid),
+ &found, nowait);
+ if (funcent)
+ {
+ pgstat_merge_funcentry(pendent, funcent, !found);
+ hash_search(pgStatPendingFunctions,
+ (void *) &(pendent->functionid), HASH_REMOVE, NULL);
+ }
+ }
+
+ /* destroy the hsah if no entry remains */
+ if (hash_get_num_entries(pgStatPendingFunctions) == 0)
+ {
+ hash_destroy(pgStatPendingFunctions);
+ pgStatPendingFunctions = NULL;
+ }
+ }
+
+ if (release_db)
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * pgstat_merge_backendstats_to_funcentry: subroutine for
+ * pgstat_update_funcstats
+ *
+ * Merges BackendFunctionEntry into StatFuncEntry
+ */
+static inline void
+pgstat_merge_backendstats_to_funcentry(PgStat_StatFuncEntry *dest,
+ PgStat_BackendFunctionEntry *src,
+ bool init)
+{
+ if (init)
+ {
+ /*
+ * If it's a new function entry, initialize counters to the values
+ * we just got.
+ */
+ dest->f_numcalls = src->f_counts.f_numcalls;
+ dest->f_total_time =
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_total_time);
+ dest->f_self_time =
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_self_time);
+ }
+ else
+ {
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ dest->f_numcalls += src->f_counts.f_numcalls;
+ dest->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_total_time);
+ dest->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_self_time);
+ }
+}
+
+/*
+ * pgstat_merge_funcentry: subroutine for pgstat_update_funcstats
+ *
+ * Merges two StatFuncEntrys
+ */
+static inline void
+pgstat_merge_funcentry(PgStat_StatFuncEntry *dest, PgStat_StatFuncEntry *src,
+ bool init)
+{
+ if (init)
+ {
+ /*
+ * If it's a new function entry, initialize counters to the values
+ * we just got.
+ */
+ dest->f_numcalls = src->f_numcalls;
+ dest->f_total_time = src->f_total_time;
+ dest->f_self_time = src->f_self_time;
+ }
+ else
+ {
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ dest->f_numcalls += src->f_numcalls;
+ dest->f_total_time += src->f_total_time;
+ dest->f_self_time += src->f_self_time;
+ }
+}
+
+
+
+/* ----------
+ * pgstat_vacuum_stat() -
+ *
+ * Remove objects he can get rid of.
+ * ----------
+ */
+void
+pgstat_vacuum_stat(void)
+{
+ HTAB *oidtab;
+ dshash_table *dshtable;
+ dshash_seq_status dshstat;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ PgStat_StatFuncEntry *funcentry;
+
+ /* we don't collect statistics under standalone mode */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* If not done for this transaction, take a snapshot of stats */
+ if (!backend_snapshot_global_stats())
+ return;
+
+ /*
+ * Read pg_database and make a list of OIDs of all existing databases
+ */
+ oidtab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+
+ /*
+ * Search the database hash table for dead databases and drop them
+ * from the hash.
+ */
+
+ dshash_seq_init(&dshstat, db_stats, false, true);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&dshstat)) != NULL)
+ {
+ Oid dbid = dbentry->databaseid;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* the DB entry for shared tables (with InvalidOid) is never dropped */
+ if (OidIsValid(dbid) &&
+ hash_search(oidtab, (void *) &dbid, HASH_FIND, NULL) == NULL)
+ pgstat_drop_database(dbid);
+ }
+
+ /* Clean up */
+ hash_destroy(oidtab);
+
+ /*
+ * Lookup our own database entry; if not found, nothing more to do.
+ */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ if (!dbentry)
+ return;
+
+ /*
+ * Similarly to above, make a list of all known relations in this DB.
+ */
+ oidtab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
+
+ /*
+ * Check for all tables listed in stats hashtable if they still exist.
+ * Stats cache is useless here so directly search the shared hash.
+ */
+ dshtable = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&dshstat, dshtable, false, true);
+ while ((tabentry = (PgStat_StatTabEntry *) dshash_seq_next(&dshstat)) != NULL)
+ {
+ Oid tabid = tabentry->tableid;
+
+ CHECK_FOR_INTERRUPTS();
+
+ if (hash_search(oidtab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* Not there, so purge this table */
+ dshash_delete_entry(dshtable, tabentry);
+ }
+ dshash_detach(dshtable);
+
+ /* Clean up */
+ hash_destroy(oidtab);
+
+ /*
+ * Now repeat the above steps for functions. However, we needn't bother
+ * in the common case where no function stats are being collected.
+ */
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshtable = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ oidtab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+
+ dshash_seq_init(&dshstat, dshtable, false, true);
+ while ((funcentry = (PgStat_StatFuncEntry *) dshash_seq_next(&dshstat)) != NULL)
+ {
+ Oid funcid = funcentry->functionid;
+
+ CHECK_FOR_INTERRUPTS();
+
+ if (hash_search(oidtab, (void *) &funcid, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* Not there, so remove this function */
+ dshash_delete_entry(dshtable, funcentry);
+ }
+
+ hash_destroy(oidtab);
+
+ dshash_detach(dshtable);
+ }
+ dshash_release_lock(db_stats, dbentry);
+}
+
+
+/*
+ * pgstat_collect_oids() -
+ *
+ * Collect the OIDs of all objects listed in the specified system catalog
+ * into a temporary hash table. Caller should hash_destroy the result after
+ * use. (However, we make the table in CurrentMemoryContext so that it will
+ * be freed properly in event of an error.)
+ */
+static HTAB *
+pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+{
+ HTAB *htab;
+ HASHCTL hash_ctl;
+ Relation rel;
+ HeapScanDesc scan;
+ HeapTuple tup;
+ Snapshot snapshot;
+
+ memset(&hash_ctl, 0, sizeof(hash_ctl));
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(Oid);
+ hash_ctl.hcxt = CurrentMemoryContext;
+ htab = hash_create("Temporary table of OIDs",
+ PGSTAT_TAB_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ rel = heap_open(catalogid, AccessShareLock);
+ snapshot = RegisterSnapshot(GetLatestSnapshot());
+ scan = heap_beginscan(rel, snapshot, 0, NULL);
+ while ((tup = heap_getnext(scan, ForwardScanDirection)) != NULL)
+ {
+ Oid thisoid;
+ bool isnull;
+
+ thisoid = heap_getattr(tup, anum_oid, RelationGetDescr(rel), &isnull);
+ Assert(!isnull);
+
+ CHECK_FOR_INTERRUPTS();
+
+ (void) hash_search(htab, (void *) &thisoid, HASH_ENTER, NULL);
+ }
+ heap_endscan(scan);
+ UnregisterSnapshot(snapshot);
+ heap_close(rel, AccessShareLock);
+
+ return htab;
+}
+
+
+/* ----------
+ * pgstat_drop_database() -
+ *
+ * Remove entry for the database that we just dropped.
+ *
+ * If some stats update happens after this, this entry will re-created but
+ * we will still clean the dead DB eventually via future invocations of
+ * pgstat_vacuum_stat().
+ * ----------
+ */
+void
+pgstat_drop_database(Oid databaseid)
+{
+ PgStat_StatDBEntry *dbentry;
+
+ Assert (OidIsValid(databaseid));
+ Assert(db_stats);
+
+ /*
+ * Lookup the database in the hashtable with exclusive lock.
+ */
+ dbentry = pgstat_get_db_entry(databaseid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ /*
+ * If found, remove it (along with the db statfile).
+ */
+ if (dbentry)
+ {
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(tbl);
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(tbl);
+ }
+
+ dshash_delete_entry(db_stats, (void *)dbentry);
+ }
+}
+
+
+/* ----------
+ * pgstat_reset_counters() -
+ *
+ * Reset counters for our database.
+ *
+ * Permission checking for this function is managed through the normal
+ * GRANT system.
+ * ----------
+ */
+void
+pgstat_reset_counters(void)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
+
+ Assert(db_stats);
+
+ /*
+ * Lookup the database in the hashtable. Nothing to do if not there.
+ */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, &status);
+
+ if (!dbentry)
+ return;
+
+ /*
+ * We simply throw away all the database's table entries by recreating a
+ * new hash table for them.
+ */
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(t);
+ dbentry->tables = DSM_HANDLE_INVALID;
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(t);
+ dbentry->functions = DSM_HANDLE_INVALID;
+ }
+
+ /*
+ * Reset database-level stats, too. This creates empty hash tables for
+ * tables and functions.
+ */
+ reset_dbentry_counters(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/* ----------
+ * pgstat_reset_shared_counters() -
+ *
+ * Reset cluster-wide shared counters.
+ *
+ * Permission checking for this function is managed through the normal
+ * GRANT system.
+ * ----------
+ */
+void
+pgstat_reset_shared_counters(const char *target)
+{
+ Assert(db_stats);
+
+ /* Reset the archiver statistics for the cluster. */
+ if (strcmp(target, "archiver") == 0)
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ memset(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = GetCurrentTimestamp();
+ }
+ else if (strcmp(target, "bgwriter") == 0)
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ /* Reset the global background writer statistics for the cluster. */
+ memset(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized reset target: \"%s\"", target),
+ errhint("Target must be \"archiver\" or \"bgwriter\".")));
+
+ LWLockRelease(StatsLock);
+}
+
+/* ----------
+ * pgstat_reset_single_counter() -
+ *
+ * Reset a single counter.
+ *
+ * Permission checking for this function is managed through the normal
+ * GRANT system.
+ * ----------
+ */
+void
+pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
+{
+ PgStat_StatDBEntry *dbentry;
+
+
+ Assert(db_stats);
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ if (!dbentry)
+ return;
+
+ /* Set the reset timestamp for the whole database */
+ dbentry->stat_reset_timestamp = GetCurrentTimestamp();
+
+ /* Remove object if it exists, ignore it if not */
+ if (type == RESET_TABLE)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_delete_key(t, (void *) &objoid);
+ }
+
+ if (type == RESET_FUNCTION && dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_delete_key(t, (void *) &objoid);
+ }
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * pgstat_reset_all_counters: subroutine for pgstat_reset_all
+ *
+ * clear all counters on shared memory
+ */
+static void
+pgstat_reset_all_counters(void)
+{
+ dshash_seq_status dshstat;
+ PgStat_StatDBEntry *dbentry;
+
+ Assert (db_stats);
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ dshash_seq_init(&dshstat, db_stats, false, true);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&dshstat)) != NULL)
+ {
+ /*
+ * We simply throw away all the database's table hashes
+ */
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(t);
+ dbentry->tables = DSM_HANDLE_INVALID;
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(t);
+ dbentry->functions = DSM_HANDLE_INVALID;
+ }
+
+ /*
+ * Reset database-level stats, too. This creates empty hash tables
+ * for tables and functions.
+ */
+ reset_dbentry_counters(dbentry);
+ dshash_release_lock(db_stats, dbentry);
+
+ }
+
+ /*
+ * Reset global counters
+ */
+ memset(shared_globalStats, 0, sizeof(*shared_globalStats));
+ memset(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_globalStats->stat_reset_timestamp =
+ shared_archiverStats->stat_reset_timestamp = GetCurrentTimestamp();
+
+ LWLockRelease(StatsLock);
+}
+
+/* ----------
+ * pgstat_report_autovac() -
+ *
+ * Called from autovacuum.c to report startup of an autovacuum process.
+ * We are called before InitPostgres is done, so can't rely on MyDatabaseId;
+ * the db OID must be passed in, instead.
+ * ----------
+ */
+void
+pgstat_report_autovac(Oid dboid)
+{
+ PgStat_StatDBEntry *dbentry;
+
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
+
+ /*
+ * Store the last autovacuum time in the database's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ dbentry->last_autovac_time = GetCurrentTimestamp();
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+
+/* ---------
+ * pgstat_report_vacuum() -
+ *
+ * Repot about the table we just vacuumed.
+ * ---------
+ */
+void
+pgstat_report_vacuum(Oid tableoid, bool shared,
+ PgStat_Counter livetuples, PgStat_Counter deadtuples)
+{
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
+
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
+
+ dboid = shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ table = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ tabentry = pgstat_get_tab_entry(table, tableoid, true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->vacuum_count++;
+ }
+ dshash_release_lock(table, tabentry);
+ dshash_detach(table);
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/* --------
+ * pgstat_report_analyze() -
+ *
+ * Report about the table we just analyzed.
+ *
+ * Caller must provide new live- and dead-tuples estimates, as well as a
+ * flag indicating whether to reset the changes_since_analyze counter.
+ * --------
+ */
+void
+pgstat_report_analyze(Relation rel,
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ bool resetcounter)
+{
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
+
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
+
+ /*
+ * Unlike VACUUM, ANALYZE might be running inside a transaction that has
+ * already inserted and/or deleted rows in the target table. ANALYZE will
+ * have counted such rows as live or dead respectively. Because we will
+ * report our counts of such rows at transaction end, we should subtract
+ * off these counts from what we send to the collector now, else they'll
+ * be double-counted after commit. (This approach also ensures that the
+ * collector ends up with the right numbers if we abort instead of
+ * committing.)
+ */
+ if (rel->pgstat_info != NULL)
+ {
+ PgStat_TableXactStatus *trans;
+
+ for (trans = rel->pgstat_info->trans; trans; trans = trans->upper)
+ {
+ livetuples -= trans->tuples_inserted - trans->tuples_deleted;
+ deadtuples -= trans->tuples_updated + trans->tuples_deleted;
+ }
+ /* count stuff inserted by already-aborted subxacts, too */
+ deadtuples -= rel->pgstat_info->t_counts.t_delta_dead_tuples;
+ /* Since ANALYZE's counts are estimates, we could have underflowed */
+ livetuples = Max(livetuples, 0);
+ deadtuples = Max(deadtuples, 0);
+ }
+
+ dboid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ table = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ tabentry = pgstat_get_tab_entry(table, RelationGetRelid(rel), true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+ dshash_release_lock(table, tabentry);
+ dshash_detach(table);
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/* --------
+ * pgstat_report_recovery_conflict() -
+ *
+ * Report a Hot Standby recovery conflict.
+ * --------
+ */
+static int pending_conflict_tablespace = 0;
+static int pending_conflict_lock = 0;
+static int pending_conflict_snapshot = 0;
+static int pending_conflict_bufferpin = 0;
+static int pending_conflict_startup_deadlock = 0;
+
+void
+pgstat_report_recovery_conflict(int reason)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
+
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
+
+ pgStatPendingRecoveryConflicts = true;
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ pending_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ pending_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ pending_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ pending_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ pending_conflict_startup_deadlock++;
+ break;
+ }
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ pgstat_cleanup_recovery_conflict(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * clean up function for pending recovery conflicts
+ */
+static void
+pgstat_cleanup_recovery_conflict(PgStat_StatDBEntry *dbentry)
+{
+ dbentry->n_conflict_tablespace += pending_conflict_tablespace;
+ dbentry->n_conflict_lock += pending_conflict_lock;
+ dbentry->n_conflict_snapshot += pending_conflict_snapshot;
+ dbentry->n_conflict_bufferpin += pending_conflict_bufferpin;
+ dbentry->n_conflict_startup_deadlock += pending_conflict_startup_deadlock;
+
+ pending_conflict_tablespace = 0;
+ pending_conflict_lock = 0;
+ pending_conflict_snapshot = 0;
+ pending_conflict_bufferpin = 0;
+ pending_conflict_startup_deadlock = 0;
+
+ pgStatPendingRecoveryConflicts = false;
+}
+
+/* --------
+ * pgstat_report_deadlock() -
+ *
+ * Report a deadlock detected.
+ * --------
+ */
+static int pending_deadlocks = 0;
+
+void
+pgstat_report_deadlock(void)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
+
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
+
+ pending_deadlocks++;
+ pgStatPendingDeadlocks = true;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ pgstat_cleanup_deadlock(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * clean up function for pending dead locks
+ */
+static void
+pgstat_cleanup_deadlock(PgStat_StatDBEntry *dbentry)
+{
+ dbentry->n_deadlocks += pending_deadlocks;
+ pending_deadlocks = 0;
+ pgStatPendingDeadlocks = false;
+}
+
+/* --------
+ * pgstat_report_tempfile() -
+ *
+ * Report a temporary file.
+ * --------
+ */
+static size_t pending_filesize = 0;
+static size_t pending_files = 0;
+
+void
+pgstat_report_tempfile(size_t filesize)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
+
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
+
+ if (filesize > 0) /* Is't there a case where filesize is really 0? */
+ {
+ pgStatPendingTempfiles = true;
+ pending_filesize += filesize; /* needs check overflow */
+ pending_files++;
+ }
+
+ if (!pgStatPendingTempfiles)
+ return;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ pgstat_cleanup_tempfile(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * clean up function for temporary files
+ */
+static void
+pgstat_cleanup_tempfile(PgStat_StatDBEntry *dbentry)
+{
+
+ dbentry->n_temp_bytes += pending_filesize;
+ dbentry->n_temp_files += pending_files;
+ pending_filesize = 0;
+ pending_files = 0;
+ pgStatPendingTempfiles = false;
+
+}
+
+/*
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
+ */
+void
+pgstat_init_function_usage(FunctionCallInfo fcinfo,
+ PgStat_FunctionCallUsage *fcu)
+{
+ PgStat_BackendFunctionEntry *htabent;
+ bool found;
+
+ if (pgstat_track_functions <= fcinfo->flinfo->fn_stats)
+ {
+ /* stats not wanted */
+ fcu->fs = NULL;
+ return;
+ }
+
+ if (!pgStatFunctions)
+ {
+ /* First time through - initialize function stat table */
+ HASHCTL hash_ctl;
+
+ memset(&hash_ctl, 0, sizeof(hash_ctl));
+ hash_ctl.keysize = sizeof(Oid);
+ hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
+ pgStatFunctions = hash_create("Function stat entries",
+ PGSTAT_FUNCTION_HASH_SIZE,
+ &hash_ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ /* Get the stats entry for this function, create if necessary */
+ htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
+ HASH_ENTER, &found);
+ if (!found)
+ MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+
+ fcu->fs = &htabent->f_counts;
+
+ /* save stats for this function, later used to compensate for recursion */
+ fcu->save_f_total_time = htabent->f_counts.f_total_time;
+
+ /* save current backend-wide total time */
+ fcu->save_total = total_func_time;
+
+ /* get clock time as of function start */
+ INSTR_TIME_SET_CURRENT(fcu->f_start);
+}
+
+/*
+ * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
+ * for specified function
+ *
+ * If no entry, return NULL, don't create a new one
+ */
+PgStat_BackendFunctionEntry *
+find_funcstat_entry(Oid func_id)
+{
+ if (pgStatFunctions == NULL)
+ return NULL;
+
+ return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
+ (void *) &func_id,
+ HASH_FIND, NULL);
+}
+
+/*
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ */
+void
+pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
+{
+ PgStat_FunctionCounts *fs = fcu->fs;
+ instr_time f_total;
+ instr_time f_others;
+ instr_time f_self;
+
+ /* stats not wanted? */
+ if (fs == NULL)
+ return;
+
+ /* total elapsed time in this function call */
+ INSTR_TIME_SET_CURRENT(f_total);
+ INSTR_TIME_SUBTRACT(f_total, fcu->f_start);
+
+ /* self usage: elapsed minus anything already charged to other calls */
+ f_others = total_func_time;
+ INSTR_TIME_SUBTRACT(f_others, fcu->save_total);
+ f_self = f_total;
+ INSTR_TIME_SUBTRACT(f_self, f_others);
+
+ /* update backend-wide total time */
+ INSTR_TIME_ADD(total_func_time, f_self);
+
+ /*
+ * Compute the new f_total_time as the total elapsed time added to the
+ * pre-call value of f_total_time. This is necessary to avoid
+ * double-counting any time taken by recursive calls of myself. (We do
+ * not need any similar kluge for self time, since that already excludes
+ * any recursive calls.)
+ */
+ INSTR_TIME_ADD(f_total, fcu->save_f_total_time);
+
+ /* update counters in function stats table */
+ if (finalize)
+ fs->f_numcalls++;
+ fs->f_total_time = f_total;
+ INSTR_TIME_ADD(fs->f_self_time, f_self);
+}
+
+
+/* ----------
+ * pgstat_initstats() -
+ *
+ * Initialize a relcache entry to count access statistics.
+ * Called whenever a relation is opened.
+ *
+ * We assume that a relcache entry's pgstat_info field is zeroed by
+ * relcache.c when the relcache entry is made; thereafter it is long-lived
+ * data. We can avoid repeated searches of the TabStatus arrays when the
+ * same relation is touched repeatedly within a transaction.
+ * ----------
+ */
+void
+pgstat_initstats(Relation rel)
+{
+ Oid rel_id = rel->rd_id;
+ char relkind = rel->rd_rel->relkind;
+
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ {
+ /* We're not counting at all */
+ rel->pgstat_info = NULL;
+ return;
+ }
+
+ /* We only count stats for things that have storage */
+ if (!(relkind == RELKIND_RELATION ||
+ relkind == RELKIND_MATVIEW ||
+ relkind == RELKIND_INDEX ||
+ relkind == RELKIND_TOASTVALUE ||
+ relkind == RELKIND_SEQUENCE))
+ {
+ rel->pgstat_info = NULL;
+ return;
+ }
+
+ /*
+ * If we already set up this relation in the current transaction, nothing
+ * to do.
+ */
+ if (rel->pgstat_info != NULL &&
+ rel->pgstat_info->t_id == rel_id)
+ return;
+
+ /* Else find or make the PgStat_TableStatus entry, and update link */
+ rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+}
+
+/*
+ * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+ */
+static PgStat_TableStatus *
+get_tabstat_entry(Oid rel_id, bool isshared)
+{
+ TabStatHashEntry *hash_entry;
+ PgStat_TableStatus *entry;
+ TabStatusArray *tsa;
+ bool found;
+
+ /*
+ * Create hash table if we don't have it already.
+ */
+ if (pgStatTabHash == NULL)
+ {
+ HASHCTL ctl;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(TabStatHashEntry);
+
+ pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
+ TABSTAT_QUANTUM,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ /*
+ * Find an entry or create a new one.
+ */
+ hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
+ if (!found)
+ {
+ /* initialize new entry with null pointer */
+ hash_entry->tsa_entry = NULL;
+ }
+
+ /*
+ * If entry is already valid, we're done.
+ */
+ if (hash_entry->tsa_entry)
+ return hash_entry->tsa_entry;
+
+ /*
+ * Locate the first pgStatTabList entry with free space, making a new list
+ * entry if needed. Note that we could get an OOM failure here, but if so
+ * we have left the hashtable and the list in a consistent state.
+ */
+ if (pgStatTabList == NULL)
+ {
+ /* Set up first pgStatTabList entry */
+ pgStatTabList = (TabStatusArray *)
+ MemoryContextAllocZero(TopMemoryContext,
+ sizeof(TabStatusArray));
+ }
+
+ tsa = pgStatTabList;
+ while (tsa->tsa_used >= TABSTAT_QUANTUM)
+ {
+ if (tsa->tsa_next == NULL)
+ tsa->tsa_next = (TabStatusArray *)
+ MemoryContextAllocZero(TopMemoryContext,
+ sizeof(TabStatusArray));
+ tsa = tsa->tsa_next;
+ }
+
+ /*
+ * Allocate a PgStat_TableStatus entry within this list entry. We assume
+ * the entry was already zeroed, either at creation or after last use.
+ */
+ entry = &tsa->tsa_entries[tsa->tsa_used++];
+ entry->t_id = rel_id;
+ entry->t_shared = isshared;
+
+ /*
+ * Now we can fill the entry in pgStatTabHash.
+ */
+ hash_entry->tsa_entry = entry;
+
+ return entry;
+}
+
+/*
+ * find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
+ *
+ * If no entry, return NULL, don't create a new one
+ *
+ * Note: if we got an error in the most recent execution of pgstat_report_stat,
+ * it's possible that an entry exists but there's no hashtable entry for it.
+ * That's okay, we'll treat this case as "doesn't exist".
+ */
+PgStat_TableStatus *
+find_tabstat_entry(Oid rel_id)
+{
+ TabStatHashEntry *hash_entry;
+
+ /* If hashtable doesn't exist, there are no entries at all */
+ if (!pgStatTabHash)
+ return NULL;
+
+ hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
+ if (!hash_entry)
+ return NULL;
+
+ /* Note that this step could also return NULL, but that's correct */
+ return hash_entry->tsa_entry;
+}
+
+/*
+ * get_tabstat_stack_level - add a new (sub)transaction stack entry if needed
+ */
+static PgStat_SubXactStatus *
+get_tabstat_stack_level(int nest_level)
+{
+ PgStat_SubXactStatus *xact_state;
+
+ xact_state = pgStatXactStack;
+ if (xact_state == NULL || xact_state->nest_level != nest_level)
+ {
+ xact_state = (PgStat_SubXactStatus *)
+ MemoryContextAlloc(TopTransactionContext,
+ sizeof(PgStat_SubXactStatus));
+ xact_state->nest_level = nest_level;
+ xact_state->prev = pgStatXactStack;
+ xact_state->first = NULL;
+ pgStatXactStack = xact_state;
+ }
+ return xact_state;
+}
+
+/*
+ * add_tabstat_xact_level - add a new (sub)transaction state record
+ */
+static void
+add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level)
+{
+ PgStat_SubXactStatus *xact_state;
+ PgStat_TableXactStatus *trans;
+
+ /*
+ * If this is the first rel to be modified at the current nest level, we
+ * first have to push a transaction stack entry.
+ */
+ xact_state = get_tabstat_stack_level(nest_level);
+
+ /* Now make a per-table stack entry */
+ trans = (PgStat_TableXactStatus *)
+ MemoryContextAllocZero(TopTransactionContext,
+ sizeof(PgStat_TableXactStatus));
+ trans->nest_level = nest_level;
+ trans->upper = pgstat_info->trans;
+ trans->parent = pgstat_info;
+ trans->next = xact_state->first;
+ xact_state->first = trans;
+ pgstat_info->trans = trans;
+}
+
+/*
+ * pgstat_count_heap_insert - count a tuple insertion of n tuples
+ */
+void
+pgstat_count_heap_insert(Relation rel, PgStat_Counter n)
+{
+ PgStat_TableStatus *pgstat_info = rel->pgstat_info;
+
+ if (pgstat_info != NULL)
+ {
+ /* We have to log the effect at the proper transactional level */
+ int nest_level = GetCurrentTransactionNestLevel();
+
+ if (pgstat_info->trans == NULL ||
+ pgstat_info->trans->nest_level != nest_level)
+ add_tabstat_xact_level(pgstat_info, nest_level);
+
+ pgstat_info->trans->tuples_inserted += n;
+ }
+}
+
+/*
+ * pgstat_count_heap_update - count a tuple update
+ */
+void
+pgstat_count_heap_update(Relation rel, bool hot)
+{
+ PgStat_TableStatus *pgstat_info = rel->pgstat_info;
+
+ if (pgstat_info != NULL)
+ {
+ /* We have to log the effect at the proper transactional level */
+ int nest_level = GetCurrentTransactionNestLevel();
+
+ if (pgstat_info->trans == NULL ||
+ pgstat_info->trans->nest_level != nest_level)
+ add_tabstat_xact_level(pgstat_info, nest_level);
+
+ pgstat_info->trans->tuples_updated++;
+
+ /* t_tuples_hot_updated is nontransactional, so just advance it */
+ if (hot)
+ pgstat_info->t_counts.t_tuples_hot_updated++;
+ }
+}
+
+/*
+ * pgstat_count_heap_delete - count a tuple deletion
+ */
+void
+pgstat_count_heap_delete(Relation rel)
+{
+ PgStat_TableStatus *pgstat_info = rel->pgstat_info;
+
+ if (pgstat_info != NULL)
+ {
+ /* We have to log the effect at the proper transactional level */
+ int nest_level = GetCurrentTransactionNestLevel();
+
+ if (pgstat_info->trans == NULL ||
+ pgstat_info->trans->nest_level != nest_level)
+ add_tabstat_xact_level(pgstat_info, nest_level);
+
+ pgstat_info->trans->tuples_deleted++;
+ }
+}
+
+/*
+ * pgstat_truncate_save_counters
+ *
+ * Whenever a table is truncated, we save its i/u/d counters so that they can
+ * be cleared, and if the (sub)xact that executed the truncate later aborts,
+ * the counters can be restored to the saved (pre-truncate) values. Note we do
+ * this on the first truncate in any particular subxact level only.
+ */
+static void
+pgstat_truncate_save_counters(PgStat_TableXactStatus *trans)
+{
+ if (!trans->truncated)
+ {
+ trans->inserted_pre_trunc = trans->tuples_inserted;
+ trans->updated_pre_trunc = trans->tuples_updated;
+ trans->deleted_pre_trunc = trans->tuples_deleted;
+ trans->truncated = true;
+ }
+}
+
+/*
+ * pgstat_truncate_restore_counters - restore counters when a truncate aborts
+ */
+static void
+pgstat_truncate_restore_counters(PgStat_TableXactStatus *trans)
+{
+ if (trans->truncated)
+ {
+ trans->tuples_inserted = trans->inserted_pre_trunc;
+ trans->tuples_updated = trans->updated_pre_trunc;
+ trans->tuples_deleted = trans->deleted_pre_trunc;
+ }
+}
+
+/*
+ * pgstat_count_truncate - update tuple counters due to truncate
+ */
+void
+pgstat_count_truncate(Relation rel)
+{
+ PgStat_TableStatus *pgstat_info = rel->pgstat_info;
+
+ if (pgstat_info != NULL)
+ {
+ /* We have to log the effect at the proper transactional level */
+ int nest_level = GetCurrentTransactionNestLevel();
+
+ if (pgstat_info->trans == NULL ||
+ pgstat_info->trans->nest_level != nest_level)
+ add_tabstat_xact_level(pgstat_info, nest_level);
+
+ pgstat_truncate_save_counters(pgstat_info->trans);
+ pgstat_info->trans->tuples_inserted = 0;
+ pgstat_info->trans->tuples_updated = 0;
+ pgstat_info->trans->tuples_deleted = 0;
+ }
+}
+
+/*
+ * pgstat_update_heap_dead_tuples - update dead-tuples count
+ *
+ * The semantics of this are that we are reporting the nontransactional
+ * recovery of "delta" dead tuples; so t_delta_dead_tuples decreases
+ * rather than increasing, and the change goes straight into the per-table
+ * counter, not into transactional state.
+ */
+void
+pgstat_update_heap_dead_tuples(Relation rel, int delta)
+{
+ PgStat_TableStatus *pgstat_info = rel->pgstat_info;
+
+ if (pgstat_info != NULL)
+ pgstat_info->t_counts.t_delta_dead_tuples -= delta;
+}
+
+
+/* ----------
+ * AtEOXact_PgStat
+ *
+ * Called from access/transam/xact.c at top-level transaction commit/abort.
+ * ----------
+ */
+void
+AtEOXact_PgStat(bool isCommit)
+{
+ PgStat_SubXactStatus *xact_state;
+
+ /*
+ * Count transaction commit or abort. (We use counters, not just bools,
+ * in case the reporting message isn't sent right away.)
+ */
+ if (isCommit)
+ pgStatXactCommit++;
+ else
+ pgStatXactRollback++;
+
+ /*
+ * Transfer transactional insert/update counts into the base tabstat
+ * entries. We don't bother to free any of the transactional state, since
+ * it's all in TopTransactionContext and will go away anyway.
+ */
+ xact_state = pgStatXactStack;
+ if (xact_state != NULL)
+ {
+ PgStat_TableXactStatus *trans;
+
+ Assert(xact_state->nest_level == 1);
+ Assert(xact_state->prev == NULL);
+ for (trans = xact_state->first; trans != NULL; trans = trans->next)
+ {
+ PgStat_TableStatus *tabstat;
+
+ Assert(trans->nest_level == 1);
+ Assert(trans->upper == NULL);
+ tabstat = trans->parent;
+ Assert(tabstat->trans == trans);
+ /* restore pre-truncate stats (if any) in case of aborted xact */
+ if (!isCommit)
+ pgstat_truncate_restore_counters(trans);
+ /* count attempted actions regardless of commit/abort */
+ tabstat->t_counts.t_tuples_inserted += trans->tuples_inserted;
+ tabstat->t_counts.t_tuples_updated += trans->tuples_updated;
+ tabstat->t_counts.t_tuples_deleted += trans->tuples_deleted;
+ if (isCommit)
+ {
+ tabstat->t_counts.t_truncated = trans->truncated;
+ if (trans->truncated)
+ {
+ /* forget live/dead stats seen by backend thus far */
+ tabstat->t_counts.t_delta_live_tuples = 0;
+ tabstat->t_counts.t_delta_dead_tuples = 0;
+ }
+ /* insert adds a live tuple, delete removes one */
+ tabstat->t_counts.t_delta_live_tuples +=
+ trans->tuples_inserted - trans->tuples_deleted;
+ /* update and delete each create a dead tuple */
+ tabstat->t_counts.t_delta_dead_tuples +=
+ trans->tuples_updated + trans->tuples_deleted;
+ /* insert, update, delete each count as one change event */
+ tabstat->t_counts.t_changed_tuples +=
+ trans->tuples_inserted + trans->tuples_updated +
+ trans->tuples_deleted;
+ }
+ else
+ {
+ /* inserted tuples are dead, deleted tuples are unaffected */
+ tabstat->t_counts.t_delta_dead_tuples +=
+ trans->tuples_inserted + trans->tuples_updated;
+ /* an aborted xact generates no changed_tuple events */
+ }
+ tabstat->trans = NULL;
+ }
+ }
+ pgStatXactStack = NULL;
+
+ /* mark as the next reference is the first in a transaction */
+ first_in_xact = true;
+}
+
+/* ----------
+ * AtEOSubXact_PgStat
+ *
+ * Called from access/transam/xact.c at subtransaction commit/abort.
+ * ----------
+ */
+void
+AtEOSubXact_PgStat(bool isCommit, int nestDepth)
+{
+ PgStat_SubXactStatus *xact_state;
+
+ /*
+ * Transfer transactional insert/update counts into the next higher
+ * subtransaction state.
+ */
+ xact_state = pgStatXactStack;
+ if (xact_state != NULL &&
+ xact_state->nest_level >= nestDepth)
+ {
+ PgStat_TableXactStatus *trans;
+ PgStat_TableXactStatus *next_trans;
+
+ /* delink xact_state from stack immediately to simplify reuse case */
+ pgStatXactStack = xact_state->prev;
+
+ for (trans = xact_state->first; trans != NULL; trans = next_trans)
+ {
+ PgStat_TableStatus *tabstat;
+
+ next_trans = trans->next;
+ Assert(trans->nest_level == nestDepth);
+ tabstat = trans->parent;
+ Assert(tabstat->trans == trans);
+ if (isCommit)
+ {
+ if (trans->upper && trans->upper->nest_level == nestDepth - 1)
+ {
+ if (trans->truncated)
+ {
+ /* propagate the truncate status one level up */
+ pgstat_truncate_save_counters(trans->upper);
+ /* replace upper xact stats with ours */
+ trans->upper->tuples_inserted = trans->tuples_inserted;
+ trans->upper->tuples_updated = trans->tuples_updated;
+ trans->upper->tuples_deleted = trans->tuples_deleted;
+ }
+ else
+ {
+ trans->upper->tuples_inserted += trans->tuples_inserted;
+ trans->upper->tuples_updated += trans->tuples_updated;
+ trans->upper->tuples_deleted += trans->tuples_deleted;
+ }
+ tabstat->trans = trans->upper;
+ pfree(trans);
+ }
+ else
+ {
+ /*
+ * When there isn't an immediate parent state, we can just
+ * reuse the record instead of going through a
+ * palloc/pfree pushup (this works since it's all in
+ * TopTransactionContext anyway). We have to re-link it
+ * into the parent level, though, and that might mean
+ * pushing a new entry into the pgStatXactStack.
+ */
+ PgStat_SubXactStatus *upper_xact_state;
+
+ upper_xact_state = get_tabstat_stack_level(nestDepth - 1);
+ trans->next = upper_xact_state->first;
+ upper_xact_state->first = trans;
+ trans->nest_level = nestDepth - 1;
+ }
+ }
+ else
+ {
+ /*
+ * On abort, update top-level tabstat counts, then forget the
+ * subtransaction
+ */
+
+ /* first restore values obliterated by truncate */
+ pgstat_truncate_restore_counters(trans);
+ /* count attempted actions regardless of commit/abort */
+ tabstat->t_counts.t_tuples_inserted += trans->tuples_inserted;
+ tabstat->t_counts.t_tuples_updated += trans->tuples_updated;
+ tabstat->t_counts.t_tuples_deleted += trans->tuples_deleted;
+ /* inserted tuples are dead, deleted tuples are unaffected */
+ tabstat->t_counts.t_delta_dead_tuples +=
+ trans->tuples_inserted + trans->tuples_updated;
+ tabstat->trans = trans->upper;
+ pfree(trans);
+ }
+ }
+ pfree(xact_state);
+ }
+}
+
+
+/*
+ * AtPrepare_PgStat
+ * Save the transactional stats state at 2PC transaction prepare.
+ *
+ * In this phase we just generate 2PC records for all the pending
+ * transaction-dependent stats work.
+ */
+void
+AtPrepare_PgStat(void)
+{
+ PgStat_SubXactStatus *xact_state;
+
+ xact_state = pgStatXactStack;
+ if (xact_state != NULL)
+ {
+ PgStat_TableXactStatus *trans;
+
+ Assert(xact_state->nest_level == 1);
+ Assert(xact_state->prev == NULL);
+ for (trans = xact_state->first; trans != NULL; trans = trans->next)
+ {
+ PgStat_TableStatus *tabstat;
+ TwoPhasePgStatRecord record;
+
+ Assert(trans->nest_level == 1);
+ Assert(trans->upper == NULL);
+ tabstat = trans->parent;
+ Assert(tabstat->trans == trans);
+
+ record.tuples_inserted = trans->tuples_inserted;
+ record.tuples_updated = trans->tuples_updated;
+ record.tuples_deleted = trans->tuples_deleted;
+ record.inserted_pre_trunc = trans->inserted_pre_trunc;
+ record.updated_pre_trunc = trans->updated_pre_trunc;
+ record.deleted_pre_trunc = trans->deleted_pre_trunc;
+ record.t_id = tabstat->t_id;
+ record.t_shared = tabstat->t_shared;
+ record.t_truncated = trans->truncated;
+
+ RegisterTwoPhaseRecord(TWOPHASE_RM_PGSTAT_ID, 0,
+ &record, sizeof(TwoPhasePgStatRecord));
+ }
+ }
+}
+
+/*
+ * PostPrepare_PgStat
+ * Clean up after successful PREPARE.
+ *
+ * All we need do here is unlink the transaction stats state from the
+ * nontransactional state. The nontransactional action counts will be
+ * reported to the stats collector immediately, while the effects on live
+ * and dead tuple counts are preserved in the 2PC state file.
+ *
+ * Note: AtEOXact_PgStat is not called during PREPARE.
+ */
+void
+PostPrepare_PgStat(void)
+{
+ PgStat_SubXactStatus *xact_state;
+
+ /*
+ * We don't bother to free any of the transactional state, since it's all
+ * in TopTransactionContext and will go away anyway.
+ */
+ xact_state = pgStatXactStack;
+ if (xact_state != NULL)
+ {
+ PgStat_TableXactStatus *trans;
+
+ for (trans = xact_state->first; trans != NULL; trans = trans->next)
+ {
+ PgStat_TableStatus *tabstat;
+
+ tabstat = trans->parent;
+ tabstat->trans = NULL;
+ }
+ }
+ pgStatXactStack = NULL;
+}
+
+/*
+ * 2PC processing routine for COMMIT PREPARED case.
+ *
+ * Load the saved counts into our local pgstats state.
+ */
+void
+pgstat_twophase_postcommit(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata;
+ PgStat_TableStatus *pgstat_info;
+
+ /* Find or create a tabstat entry for the rel */
+ pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+
+ /* Same math as in AtEOXact_PgStat, commit case */
+ pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
+ pgstat_info->t_counts.t_tuples_updated += rec->tuples_updated;
+ pgstat_info->t_counts.t_tuples_deleted += rec->tuples_deleted;
+ pgstat_info->t_counts.t_truncated = rec->t_truncated;
+ if (rec->t_truncated)
+ {
+ /* forget live/dead stats seen by backend thus far */
+ pgstat_info->t_counts.t_delta_live_tuples = 0;
+ pgstat_info->t_counts.t_delta_dead_tuples = 0;
+ }
+ pgstat_info->t_counts.t_delta_live_tuples +=
+ rec->tuples_inserted - rec->tuples_deleted;
+ pgstat_info->t_counts.t_delta_dead_tuples +=
+ rec->tuples_updated + rec->tuples_deleted;
+ pgstat_info->t_counts.t_changed_tuples +=
+ rec->tuples_inserted + rec->tuples_updated +
+ rec->tuples_deleted;
+}
+
+/*
+ * 2PC processing routine for ROLLBACK PREPARED case.
+ *
+ * Load the saved counts into our local pgstats state, but treat them
+ * as aborted.
+ */
+void
+pgstat_twophase_postabort(TransactionId xid, uint16 info,
+ void *recdata, uint32 len)
+{
+ TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata;
+ PgStat_TableStatus *pgstat_info;
+
+ /* Find or create a tabstat entry for the rel */
+ pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+
+ /* Same math as in AtEOXact_PgStat, abort case */
+ if (rec->t_truncated)
+ {
+ rec->tuples_inserted = rec->inserted_pre_trunc;
+ rec->tuples_updated = rec->updated_pre_trunc;
+ rec->tuples_deleted = rec->deleted_pre_trunc;
+ }
+ pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
+ pgstat_info->t_counts.t_tuples_updated += rec->tuples_updated;
+ pgstat_info->t_counts.t_tuples_deleted += rec->tuples_deleted;
+ pgstat_info->t_counts.t_delta_dead_tuples +=
+ rec->tuples_inserted + rec->tuples_updated;
+}
+
+/* ----------
+ * pgstat_fetch_stat_tabentry() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the collected statistics for one table or NULL. NULL doesn't mean
+ * that the table doesn't exist, it is just not yet known by the
+ * collector, so the caller is better off to report ZERO instead.
+ * ----------
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry(Oid relid)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+
+ /* Lookup our database, then look in its table hash table. */
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId, false);
+ if (dbentry == NULL)
+ return NULL;
+
+ tabentry = backend_get_tab_entry(dbentry, relid, false);
+ if (tabentry != NULL)
+ return tabentry;
+
+ /*
+ * If we didn't find it, maybe it's a shared table.
+ */
+ dbentry = pgstat_fetch_stat_dbentry(InvalidOid, false);
+ if (dbentry == NULL)
+ return NULL;
+
+ tabentry = backend_get_tab_entry(dbentry, relid, false);
+ if (tabentry != NULL)
+ return tabentry;
+
+ return NULL;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_funcentry() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * the collected statistics for one function or NULL.
+ * ----------
+ */
+PgStat_StatFuncEntry *
+pgstat_fetch_stat_funcentry(Oid func_id)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatFuncEntry *funcentry = NULL;
+
+ /* Lookup our database, then find the requested function */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_SHARED, NULL);
+ if (dbentry == NULL)
+ return NULL;
+
+ funcentry = backend_get_func_etnry(dbentry, func_id, false);
+
+ dshash_release_lock(db_stats, dbentry);
+ return funcentry;
+}
+
+/*
+ * ---------
+ * pgstat_fetch_stat_archiver() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * a pointer to the archiver statistics struct.
+ * ---------
+ */
+PgStat_ArchiverStats *
+pgstat_fetch_stat_archiver(void)
+{
+ /* If not done for this transaction, take a stats snapshot */
+ if (!backend_snapshot_global_stats())
+ return NULL;
+
+ return snapshot_archiverStats;
+}
+
+
+/*
+ * ---------
+ * pgstat_fetch_global() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * a pointer to the global statistics struct.
+ * ---------
+ */
+PgStat_GlobalStats *
+pgstat_fetch_global(void)
+{
+ /* If not done for this transaction, take a stats snapshot */
+ if (!backend_snapshot_global_stats())
+ return NULL;
+
+ return snapshot_globalStats;
+}
+
+/*
+ * Shut down a single backend's statistics reporting at process exit.
+ *
+ * Flush any remaining statistics counts out to the collector.
+ * Without this, operations triggered during backend exit (such as
+ * temp table deletions) won't be counted.
+ *
+ * Lastly, clear out our entry in the PgBackendStatus array.
+ */
+static void
+pgstat_beshutdown_hook(int code, Datum arg)
+{
+ /*
+ * If we got as far as discovering our own database ID, we can report what
+ * we did to the collector. Otherwise, we'd be sending an invalid
+ * database ID, so forget it. (This means that accesses to pg_database
+ * during failed backend starts might never get counted.)
+ */
+ if (OidIsValid(MyDatabaseId))
+ pgstat_update_stat(true);
+}
+
+
+/* ------------------------------------------------------------
+ * Local support functions follow
+ * ------------------------------------------------------------
+ */
+
+/* ----------
+ * pgstat_update_archiver() -
+ *
+ * Update the stats data about the WAL file that we successfully archived or
+ * failed to archive.
+ * ----------
+ */
+void
+pgstat_update_archiver(const char *xlog, bool failed)
+{
+ if (failed)
+ {
+ /* Failed archival attempt */
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = GetCurrentTimestamp();
+ }
+ else
+ {
+ /* Successful archival operation */
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = GetCurrentTimestamp();
+ }
+}
+
+/* ----------
+ * pgstat_update_bgwriter() -
+ *
+ * Update bgwriter statistics
+ * ----------
+ */
+void
+pgstat_update_bgwriter(void)
+{
+ /* We assume this initializes to zeroes */
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *s = &BgWriterStats;
+
+ /*
+ * This function can be called even if nothing at all has happened. In
+ * this case, avoid sending a completely empty message to the stats
+ * collector.
+ */
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
+ return;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += s->m_timed_checkpoints;
+ shared_globalStats->requested_checkpoints += s->m_requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += s->m_checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += s->m_checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += s->m_buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += s->m_buf_written_clean;
+ shared_globalStats->maxwritten_clean += s->m_maxwritten_clean;
+ shared_globalStats->buf_written_backend += s->m_buf_written_backend;
+ shared_globalStats->buf_fsync_backend += s->m_buf_fsync_backend;
+ shared_globalStats->buf_alloc += s->m_buf_alloc;
+ LWLockRelease(StatsLock);
+
+ /*
+ * Clear out the statistics buffer, so it can be re-used.
+ */
+ MemSet(&BgWriterStats, 0, sizeof(BgWriterStats));
+}
+
+/*
+ * Subroutine to reset stats in a shared database entry
+ *
+ * Tables and functions hashes are initialized to empty.
+ */
+static void
+reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
+{
+ dshash_table *tbl;
+
+ dbentry->n_xact_commit = 0;
+ dbentry->n_xact_rollback = 0;
+ dbentry->n_blocks_fetched = 0;
+ dbentry->n_blocks_hit = 0;
+ dbentry->n_tuples_returned = 0;
+ dbentry->n_tuples_fetched = 0;
+ dbentry->n_tuples_inserted = 0;
+ dbentry->n_tuples_updated = 0;
+ dbentry->n_tuples_deleted = 0;
+ dbentry->last_autovac_time = 0;
+ dbentry->n_conflict_tablespace = 0;
+ dbentry->n_conflict_lock = 0;
+ dbentry->n_conflict_snapshot = 0;
+ dbentry->n_conflict_bufferpin = 0;
+ dbentry->n_conflict_startup_deadlock = 0;
+ dbentry->n_temp_files = 0;
+ dbentry->n_temp_bytes = 0;
+ dbentry->n_deadlocks = 0;
+ dbentry->n_block_read_time = 0;
+ dbentry->n_block_write_time = 0;
+
+ dbentry->stat_reset_timestamp = GetCurrentTimestamp();
+ dbentry->stats_timestamp = 0;
+
+
+ Assert(dbentry->tables == DSM_HANDLE_INVALID);
+ tbl = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables = dshash_get_hash_table_handle(tbl);
+ dshash_detach(tbl);
+
+ Assert(dbentry->functions == DSM_HANDLE_INVALID);
+ /* we create function hash as needed */
+
+ dbentry->snapshot_tables = NULL;
+ dbentry->snapshot_functions = NULL;
+}
+
+/*
+ * Lookup the hash table entry for the specified database. If no hash
+ * table entry exists, initialize it, if the create parameter is true.
+ * Else, return NULL.
+ */
+static PgStat_StatDBEntry *
+pgstat_get_db_entry(Oid databaseid, int op, PgStat_TableLookupState *status)
+{
+ PgStat_StatDBEntry *result;
+ bool nowait = ((op & PGSTAT_FETCH_NOWAIT) != 0);
+ bool lock_acquired = true;
+ bool found = true;
+
+ if (!IsUnderPostmaster)
+ return NULL;
+
+ /* Lookup or create the hash table entry for this database */
+ if (op & PGSTAT_FETCH_EXCLUSIVE)
+ {
+ result = (PgStat_StatDBEntry *)
+ dshash_find_or_insert_extended(db_stats, &databaseid,
+ &found, nowait);
+ if (result == NULL)
+ lock_acquired = false;
+ else if (!found)
+ {
+ /*
+ * If not found, initialize the new one. This creates empty hash
+ * tables for tables and functions, too.
+ */
+ reset_dbentry_counters(result);
+ }
+ }
+ else
+ {
+ result = (PgStat_StatDBEntry *)
+ dshash_find_extended(db_stats, &databaseid, true, nowait,
+ &lock_acquired);
+ if (result == NULL)
+ found = false;
+ }
+
+ /* Set return status if requested */
+ if (status)
+ {
+ if (!lock_acquired)
+ {
+ Assert(nowait);
+ *status = PGSTAT_ENTRY_LOCK_FAILED;
+ }
+ else if (!found)
+ *status = PGSTAT_ENTRY_NOT_FOUND;
+ else
+ *status = PGSTAT_ENTRY_FOUND;
+ }
+
+ return result;
+}
+
+/*
+ * Lookup the hash table entry for the specified table. If no hash
+ * table entry exists, initialize it, if the create parameter is true.
+ * Else, return NULL.
+ */
+static PgStat_StatTabEntry *
+pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create)
+{
+ PgStat_StatTabEntry *result;
+ bool found;
+
+ /* Lookup or create the hash table entry for this table */
+ if (create)
+ result = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(table, &tableoid, &found);
+ else
+ result = (PgStat_StatTabEntry *) dshash_find(table, &tableoid, false);
+
+ if (!create && !found)
+ return NULL;
+
+ /* If not found, initialize the new one. */
+ if (!found)
+ {
+ result->numscans = 0;
+ result->tuples_returned = 0;
+ result->tuples_fetched = 0;
+ result->tuples_inserted = 0;
+ result->tuples_updated = 0;
+ result->tuples_deleted = 0;
+ result->tuples_hot_updated = 0;
+ result->n_live_tuples = 0;
+ result->n_dead_tuples = 0;
+ result->changes_since_analyze = 0;
+ result->blocks_fetched = 0;
+ result->blocks_hit = 0;
+ result->vacuum_timestamp = 0;
+ result->vacuum_count = 0;
+ result->autovac_vacuum_timestamp = 0;
+ result->autovac_vacuum_count = 0;
+ result->analyze_timestamp = 0;
+ result->analyze_count = 0;
+ result->autovac_analyze_timestamp = 0;
+ result->autovac_analyze_count = 0;
+ }
+
+ return result;
+}
+
+
+/* ----------
+ * pgstat_write_statsfiles() -
+ * Write the global statistics file, as well as DB files.
+ * ----------
+ */
+void
+pgstat_write_statsfiles(void)
+{
+ dshash_seq_status hstat;
+ PgStat_StatDBEntry *dbentry;
+ FILE *fpout;
+ int32 format_id;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+ int rc;
+
+ /* should be called from postmaster */
+ Assert(!IsUnderPostmaster);
+
+ elog(DEBUG2, "writing stats file \"%s\"", statfile);
+
+ /*
+ * Open the statistics temp file to write out the current values.
+ */
+ fpout = AllocateFile(tmpfile, PG_BINARY_W);
+ if (fpout == NULL)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open temporary statistics file \"%s\": %m",
+ tmpfile)));
+ return;
+ }
+
+ /*
+ * Set the timestamp of the stats file.
+ */
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
+
+ /*
+ * Write the file header --- currently just a format ID.
+ */
+ format_id = PGSTAT_FILE_FORMAT_ID;
+ rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Write global stats struct
+ */
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Write archiver stats struct
+ */
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Walk through the database table.
+ */
+ dshash_seq_init(&hstat, db_stats, false, false);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&hstat)) != NULL)
+ {
+ /*
+ * Write out the table and function stats for this DB into the
+ * appropriate per-DB stat file, if required.
+ */
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
+
+ pgstat_write_db_statsfile(dbentry);
+
+ /*
+ * Write out the DB entry. We don't write the tables or functions
+ * pointers, since they're of no use to any other process.
+ */
+ fputc('D', fpout);
+ rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+
+ /*
+ * No more output to be done. Close the temp file and replace the old
+ * pgstat.stat with it. The ferror() check replaces testing for error
+ * after each individual fputc or fwrite above.
+ */
+ fputc('E', fpout);
+
+ if (ferror(fpout))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write temporary statistics file \"%s\": %m",
+ tmpfile)));
+ FreeFile(fpout);
+ unlink(tmpfile);
+ }
+ else if (FreeFile(fpout) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close temporary statistics file \"%s\": %m",
+ tmpfile)));
+ unlink(tmpfile);
+ }
+ else if (rename(tmpfile, statfile) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
+ tmpfile, statfile)));
+ unlink(tmpfile);
+ }
+}
+
+/*
+ * return the filename for a DB stat file; filename is the output buffer,
+ * of length len.
+ */
+static void
+get_dbstat_filename(bool tempname, Oid databaseid,
+ char *filename, int len)
+{
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
+}
+
+/* ----------
+ * pgstat_write_db_statsfile() -
+ * Write the stat file for a single database.
+ *
+ * If writing to the permanent file (happens when the collector is
+ * shutting down only), remove the temporary file so that backends
+ * starting up under a new postmaster can't read the old data before
+ * the new collector is ready.
+ * ----------
+ */
+static void
+pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry)
+{
+ dshash_seq_status tstat;
+ dshash_seq_status fstat;
+ PgStat_StatTabEntry *tabentry;
+ PgStat_StatFuncEntry *funcentry;
+ FILE *fpout;
+ int32 format_id;
+ Oid dbid = dbentry->databaseid;
+ int rc;
+ char tmpfile[MAXPGPATH];
+ char statfile[MAXPGPATH];
+ dshash_table *tbl;
+
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
+
+ elog(DEBUG2, "writing stats file \"%s\"", statfile);
+
+ /*
+ * Open the statistics temp file to write out the current values.
+ */
+ fpout = AllocateFile(tmpfile, PG_BINARY_W);
+ if (fpout == NULL)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open temporary statistics file \"%s\": %m",
+ tmpfile)));
+ return;
+ }
+
+ /*
+ * Write the file header --- currently just a format ID.
+ */
+ format_id = PGSTAT_FILE_FORMAT_ID;
+ rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Walk through the database's access stats per table.
+ */
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&tstat, tbl, false, false);
+ while ((tabentry = (PgStat_StatTabEntry *) dshash_seq_next(&tstat)) != NULL)
+ {
+ fputc('T', fpout);
+ rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ dshash_detach(tbl);
+
+ /*
+ * Walk through the database's function stats table.
+ */
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ tbl = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_seq_init(&fstat, tbl, false, false);
+ while ((funcentry = (PgStat_StatFuncEntry *) dshash_seq_next(&fstat)) != NULL)
+ {
+ fputc('F', fpout);
+ rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ dshash_detach(tbl);
+ }
+
+ /*
+ * No more output to be done. Close the temp file and replace the old
+ * pgstat.stat with it. The ferror() check replaces testing for error
+ * after each individual fputc or fwrite above.
+ */
+ fputc('E', fpout);
+
+ if (ferror(fpout))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write temporary statistics file \"%s\": %m",
+ tmpfile)));
+ FreeFile(fpout);
+ unlink(tmpfile);
+ }
+ else if (FreeFile(fpout) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close temporary statistics file \"%s\": %m",
+ tmpfile)));
+ unlink(tmpfile);
+ }
+ else if (rename(tmpfile, statfile) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
+ tmpfile, statfile)));
+ unlink(tmpfile);
+ }
+}
+
+/* ----------
+ * pgstat_read_statsfiles() -
+ *
+ * Reads in some existing statistics collector files into the shared stats
+ * hash.
+ *
+ * ----------
+ */
+void
+pgstat_read_statsfiles(void)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatDBEntry dbbuf;
+ FILE *fpin;
+ int32 format_id;
+ bool found;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+ dshash_table *tblstats = NULL;
+ dshash_table *funcstats = NULL;
+
+ /* should be called from postmaster */
+ Assert(!IsUnderPostmaster);
+
+ /*
+ * local cache lives in pgStatLocalContext.
+ */
+ pgstat_setup_memcxt();
+
+ /*
+ * Create the DB hashtable and global stas area
+ */
+ /* Hold lock so that no other process looks empty stats */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ pgstat_create_shared_stats();
+
+ /*
+ * Set the current timestamp (will be kept only in case we can't load an
+ * existing statsfile).
+ */
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp = shared_globalStats->stat_reset_timestamp;
+
+ /*
+ * Try to open the stats file. If it doesn't exist, the backends simply
+ * return zero for anything and the collector simply starts from scratch
+ * with empty counters.
+ *
+ * ENOENT is a possibility if the stats collector is not running or has
+ * not yet written the stats file the first time. Any other failure
+ * condition is suspicious.
+ */
+ if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open statistics file \"%s\": %m",
+ statfile)));
+ LWLockRelease(StatsLock);
+ return;
+ }
+
+ /*
+ * Verify it's of the expected format.
+ */
+ if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
+ format_id != PGSTAT_FILE_FORMAT_ID)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
+
+ /*
+ * Read global stats struct
+ */
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ memset(shared_globalStats, 0, sizeof(*shared_globalStats));
+ goto done;
+ }
+
+ /*
+ * In the collector, disregard the timestamp we read from the permanent
+ * stats file; we should be willing to write a temp stats file immediately
+ * upon the first request from any backend. This only matters if the old
+ * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
+ * an unusual scenario.
+ */
+ shared_globalStats->stats_timestamp = 0;
+
+ /*
+ * Read archiver stats struct
+ */
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ memset(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ goto done;
+ }
+
+ /*
+ * We found an existing collector stats file. Read it and put all the
+ * hashtable entries into place.
+ */
+ for (;;)
+ {
+ switch (fgetc(fpin))
+ {
+ /*
+ * 'D' A PgStat_StatDBEntry struct describing a database
+ * follows.
+ */
+ case 'D':
+ if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
+ fpin) != offsetof(PgStat_StatDBEntry, tables))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ /*
+ * Add to the DB hash
+ */
+ dbentry = (PgStat_StatDBEntry *)
+ dshash_find_or_insert(db_stats, (void *) &dbbuf.databaseid,
+ &found);
+ if (found)
+ {
+ dshash_release_lock(db_stats, dbentry);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
+ dbentry->tables = DSM_HANDLE_INVALID;
+ dbentry->functions = DSM_HANDLE_INVALID;
+ dbentry->snapshot_tables = NULL;
+ dbentry->snapshot_functions = NULL;
+
+ /*
+ * In the collector, disregard the timestamp we read from the
+ * permanent stats file; we should be willing to write a temp
+ * stats file immediately upon the first request from any
+ * backend.
+ */
+ dbentry->stats_timestamp = 0;
+
+ /*
+ * If requested, read the data from the database-specific
+ * file. Otherwise we just leave the hashtables empty.
+ */
+ tblstats = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables = dshash_get_hash_table_handle(tblstats);
+ /* we don't create function hash at the present */
+ dshash_release_lock(db_stats, dbentry);
+ pgstat_read_db_statsfile(dbentry->databaseid,
+ tblstats, funcstats);
+ dshash_detach(tblstats);
+ break;
+
+ case 'E':
+ goto done;
+
+ default:
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+ }
+
+done:
+ LWLockRelease(StatsLock);
+ FreeFile(fpin);
+
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
+
+ return;
+}
+
+
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+void
+StatsShmemInit(void)
+{
+ bool found;
+
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+
+ /* Load saved data if any */
+ pgstat_read_statsfiles();
+
+ /* need to be called before dsm shutodwn */
+ before_shmem_exit(pgstat_postmaster_shutdown, (Datum) 0);
+ }
+}
+
+static void
+pgstat_postmaster_shutdown(int code, Datum arg)
+{
+ /* we trash the stats on crash */
+ if (code == 0)
+ pgstat_write_statsfiles();
+}
+
+/* ----------
+ * pgstat_read_db_statsfile() -
+ *
+ * Reads in the permanent statistics collector file and create shared
+ * statistics tables. The file is removed afer reading.
+ * ----------
+ */
+static void
+pgstat_read_db_statsfile(Oid databaseid,
+ dshash_table *tabhash, dshash_table *funchash)
+{
+ PgStat_StatTabEntry *tabentry;
+ PgStat_StatTabEntry tabbuf;
+ PgStat_StatFuncEntry funcbuf;
+ PgStat_StatFuncEntry *funcentry;
+ FILE *fpin;
+ int32 format_id;
+ bool found;
+ char statfile[MAXPGPATH];
+
+ /* should be called from postmaster */
+ Assert(!IsUnderPostmaster);
+
+ get_dbstat_filename(false, databaseid, statfile, MAXPGPATH);
+
+ /*
+ * Try to open the stats file. If it doesn't exist, the backends simply
+ * return zero for anything and the collector simply starts from scratch
+ * with empty counters.
+ *
+ * ENOENT is a possibility if the stats collector is not running or has
+ * not yet written the stats file the first time. Any other failure
+ * condition is suspicious.
+ */
+ if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open statistics file \"%s\": %m",
+ statfile)));
+ return;
+ }
+
+ /*
+ * Verify it's of the expected format.
+ */
+ if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
+ format_id != PGSTAT_FILE_FORMAT_ID)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
+
+ /*
+ * We found an existing collector stats file. Read it and put all the
+ * hashtable entries into place.
+ */
+ for (;;)
+ {
+ switch (fgetc(fpin))
+ {
+ /*
+ * 'T' A PgStat_StatTabEntry follows.
+ */
+ case 'T':
+ if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
+ fpin) != sizeof(PgStat_StatTabEntry))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ /*
+ * Skip if table data not wanted.
+ */
+ if (tabhash == NULL)
+ break;
+
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(tabhash,
+ (void *) &tabbuf.tableid, &found);
+
+ if (found)
+ {
+ dshash_release_lock(tabhash, tabentry);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ memcpy(tabentry, &tabbuf, sizeof(tabbuf));
+ dshash_release_lock(tabhash, tabentry);
+ break;
+
+ /*
+ * 'F' A PgStat_StatFuncEntry follows.
+ */
+ case 'F':
+ if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
+ fpin) != sizeof(PgStat_StatFuncEntry))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ /*
+ * Skip if function data not wanted.
+ */
+ if (funchash == NULL)
+ break;
+
+ funcentry = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert(funchash,
+ (void *) &funcbuf.functionid, &found);
+
+ if (found)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ memcpy(funcentry, &funcbuf, sizeof(funcbuf));
+ dshash_release_lock(funchash, funcentry);
+ break;
+
+ /*
+ * 'E' The EOF marker of a complete stats file.
+ */
+ case 'E':
+ goto done;
+
+ default:
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+ }
+
+done:
+ FreeFile(fpin);
+
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
+}
+
+/* ----------
+ * pgstat_clean_snapshot: clean up the local cache that will cause new
+ * snapshots to bo read.
+ * ----------
+ */
+void
+pgstat_clear_snapshot(void)
+{
+ Assert(pgStatLocalContext);
+ MemoryContextReset(pgStatLocalContext);
+
+ /* mark as the resource are not allocated */
+ snapshot_globalStats = NULL;
+ snapshot_archiverStats = NULL;
+ snapshot_db_stats = NULL;
+}
+
+/*
+ * create_local_stats_hash() -
+ *
+ * Creates a dynahash used for table/function stats cache.
+ */
+static HTAB *
+create_local_stats_hash(const char *name, size_t keysize, size_t entrysize,
+ int nentries)
+{
+ HTAB *result;
+ HASHCTL ctl;
+
+ /* Create the hash in the stats context */
+ ctl.keysize = keysize;
+ ctl.entrysize = entrysize;
+ ctl.hcxt = pgStatLocalContext;
+ result = hash_create(name, nentries, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ return result;
+}
+
+/*
+ * snapshot_statentry() - Find an entry from source dshash.
+ *
+ * Returns the entry for key or NULL if not found. If dest is not null, uses
+ * *dest as local cache, which is created in the same shape with the given
+ * dshash when *dest is NULL. In both cases the result is cached in the hash
+ * and the same entry is returned to subsequent calls for the same key.
+ *
+ * Otherwise returned entry is a copy that is palloc'ed in caller's memory
+ * context. Its content may differ for every request.
+ *
+ * If dshash is NULL, temporaralily attaches dsh_handle instead.
+ */
+static void *
+snapshot_statentry(HTAB **dest, const char *hashname,
+ dshash_table *dshash, dshash_table_handle dsh_handle,
+ const dshash_parameters *dsh_params, Oid key)
+{
+ char *lentry = NULL;
+ size_t keysize = dsh_params->key_size;
+ size_t entrysize = dsh_params->entry_size;
+
+ if (dest)
+ {
+ /* caches the result entry */
+ bool found;
+ bool *negative;
+
+ /*
+ * Create new hash with arbitrary initial entries since we don't know
+ * how this hash will grow.
+ */
+ if (!*dest)
+ {
+ /* make room for negative flag at the end of entry */
+ *dest = create_local_stats_hash(hashname, keysize,
+ entrysize + sizeof(bool), 32);
+ }
+
+ lentry = hash_search(*dest, &key, HASH_ENTER, &found);
+
+ /* negative flag is placed at the end of the entry */
+ negative = (bool *) (lentry + entrysize);
+
+ if (!found)
+ {
+ /* not found in local cache, search shared hash */
+
+ dshash_table *t = dshash;
+ void *sentry;
+
+ /* attach shared hash if not given */
+ if (!t)
+ t = dshash_attach(area, dsh_params, dsh_handle, NULL);
+
+ sentry = dshash_find(t, &key, false);
+
+ /*
+ * We expect that the stats for specified database exists in most
+ * cases.
+ */
+
+ if (sentry)
+ {
+ memcpy(lentry, sentry, entrysize);
+ dshash_release_lock(t, sentry);
+ }
+
+ *negative = !sentry;
+
+ /* Release it if we attached it here */
+ if (!dshash)
+ dshash_detach(t);
+
+ if (!sentry)
+ return NULL;
+ }
+
+ if (*negative)
+ lentry = NULL;
+ }
+ else
+ {
+ /*
+ * The caller don't want caching. Just make a copy of the entry then
+ * return.
+ */
+ dshash_table *t = dshash;
+ void *sentry;
+
+ if (!t)
+ t = dshash_attach(area, dsh_params, dsh_handle, NULL);
+
+ sentry = dshash_find(t, &key, false);
+ if (sentry)
+ {
+ lentry = palloc(entrysize);
+ memcpy(lentry, sentry, entrysize);
+ dshash_release_lock(t, sentry);
+ }
+
+ if (!dshash)
+ dshash_detach(t);
+ }
+
+ return (void *) lentry;
+}
+
+/*
+ * backend_snapshot_global_stats() -
+ *
+ * Makes a local copy of global stats if not already done. They will be kept
+ * until pgstat_clear_snapshot() is called or the end of the current memory
+ * context (typically TopTransactionContext). Returns false if the shared
+ * stats is not created yet.
+ */
+static bool
+backend_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext = CurrentMemoryContext;
+ TimestampTz update_time = 0;
+
+
+ /*
+ * This is the first call in a transaction. If we find the shared stats
+ * updated, throw away the cache.
+ */
+ if (IsTransactionState() && first_in_xact)
+ {
+ first_in_xact = false;
+ LWLockAcquire(StatsLock, LW_SHARED);
+ update_time = StatsShmem->last_update;
+ LWLockRelease(StatsLock);
+
+ if (backend_cache_expire < update_time)
+ {
+ pgstat_clear_snapshot();
+
+ /*
+ * Shared stats are updated frequently when many backends are
+ * running, but we don't want the cached stats to be expired so
+ * frequently. Keep them at least for the same duration with
+ * minimul stats update interval of a backend.
+ */
+ backend_cache_expire =
+ update_time + PGSTAT_STAT_MIN_INTERVAL * USECS_PER_SEC / 1000;
+ }
+ }
+
+ /* Nothing to do if already done */
+ if (snapshot_globalStats)
+ return true;
+
+ Assert(snapshot_archiverStats == NULL);
+
+ /*
+ * The snapshot lives within the current top transaction if any, or the
+ * current memory context liftime otherwise.
+ */
+ if (IsTransactionState())
+ oldcontext = MemoryContextSwitchTo(pgStatLocalContext);
+
+ /* global stats can be just copied */
+ LWLockAcquire(StatsLock, LW_SHARED);
+ snapshot_globalStats = palloc(sizeof(PgStat_GlobalStats));
+ memcpy(snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ snapshot_archiverStats = palloc(sizeof(PgStat_ArchiverStats));
+ memcpy(snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ LWLockRelease(StatsLock);
+
+ /* set the timestamp of this snapshot */
+ snapshot_globalStats->stats_timestamp = update_time;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return true;
+}
+
+/* ----------
+ * pgstat_fetch_stat_dbentry() -
+ *
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end. If onshot is true, they are not cached and returned
+ * in a palloc'ed memory in caller's context.
+ */
+PgStat_StatDBEntry *
+pgstat_fetch_stat_dbentry(Oid dbid, bool oneshot)
+{
+ /* take a local snapshot if we don't have one */
+ char *hashname = "local database stats hash";
+ PgStat_StatDBEntry *dbentry;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* If not done for this transaction, take a snapshot of global stats */
+ if (!backend_snapshot_global_stats())
+ return NULL;
+
+ dbentry = snapshot_statentry(oneshot ? NULL : &snapshot_db_stats,
+ hashname, db_stats, 0, &dsh_dbparams,
+ dbid);
+
+ return dbentry;
+}
+
+/* ----------
+ * backend_get_tab_entry() -
+ *
+ * Find table stats entry on backends. The returned entries are cached until
+ * transaction end. If onshot is true, they are not cached and returned in a
+ * palloc'ed memory in caller's context.
+ */
+PgStat_StatTabEntry *
+backend_get_tab_entry(PgStat_StatDBEntry *dbent, Oid reloid, bool oneshot)
+{
+ /* take a local snapshot if we don't have one */
+ char *hashname = "local table stats hash";
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return snapshot_statentry(oneshot ? NULL : &dbent->snapshot_tables,
+ hashname, NULL, dbent->tables, &dsh_tblparams,
+ reloid);
+}
+
+/* ----------
+ * backend_get_func_entry() -
+ *
+ * Find function stats entry on backends. The returned entries are cached
+ * until transaction end. If onshot is true, they are not cached and returned
+ * in a palloc'ed memory in caller's context.
+ */
+static PgStat_StatFuncEntry *
+backend_get_func_etnry(PgStat_StatDBEntry *dbent, Oid funcid, bool oneshot)
+{
+ char *hashname = "local table stats hash";
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ if (dbent->functions == DSM_HANDLE_INVALID)
+ return NULL;
+
+ return snapshot_statentry(oneshot ? NULL : &dbent->snapshot_tables,
+ hashname, NULL, dbent->functions, &dsh_funcparams,
+ funcid);
+}
+
+static bool
+pgstat_update_tabentry(dshash_table *tabhash, PgStat_TableStatus *stat,
+ bool nowait)
+{
+ PgStat_StatTabEntry *tabentry;
+ bool found;
+
+ if (tabhash == NULL)
+ return false;
+
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert_extended(tabhash, (void *) &(stat->t_id),
+ &found, nowait);
+
+ /* failed to acquire lock */
+ if (tabentry == NULL)
+ return false;
+
+ if (!found)
+ {
+ /*
+ * If it's a new table entry, initialize counters to the values we
+ * just got.
+ */
+ tabentry->numscans = stat->t_counts.t_numscans;
+ tabentry->tuples_returned = stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched = stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted = stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated = stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted = stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated = stat->t_counts.t_tuples_hot_updated;
+ tabentry->n_live_tuples = stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples = stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze = stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched = stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit = stat->t_counts.t_blocks_hit;
+
+ tabentry->vacuum_timestamp = 0;
+ tabentry->vacuum_count = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_count = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->analyze_count = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ tabentry->autovac_analyze_count = 0;
+ }
+ else
+ {
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ tabentry->numscans += stat->t_counts.t_numscans;
+ tabentry->tuples_returned += stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched += stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted += stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated += stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted += stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated += stat->t_counts.t_tuples_hot_updated;
+ /* If table was truncated, first reset the live/dead counters */
+ if (stat->t_counts.t_truncated)
+ {
+ tabentry->n_live_tuples = 0;
+ tabentry->n_dead_tuples = 0;
+ }
+ tabentry->n_live_tuples += stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples += stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze += stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched += stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit += stat->t_counts.t_blocks_hit;
+ }
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
+
+ dshash_release_lock(tabhash, tabentry);
+
+ return true;
+}
+
+static void
+pgstat_update_dbentry(PgStat_StatDBEntry *dbentry, PgStat_TableStatus *stat)
+{
+ /*
+ * Add per-table stats to the per-database entry, too.
+ */
+ dbentry->n_tuples_returned += stat->t_counts.t_tuples_returned;
+ dbentry->n_tuples_fetched += stat->t_counts.t_tuples_fetched;
+ dbentry->n_tuples_inserted += stat->t_counts.t_tuples_inserted;
+ dbentry->n_tuples_updated += stat->t_counts.t_tuples_updated;
+ dbentry->n_tuples_deleted += stat->t_counts.t_tuples_deleted;
+ dbentry->n_blocks_fetched += stat->t_counts.t_blocks_fetched;
+ dbentry->n_blocks_hit += stat->t_counts.t_blocks_hit;
+}
+
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgStatLocalContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgStatLocalContext)
+ pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
+ "Activity statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+}
+
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 273e2f385f..4edd980ffc 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -34,6 +34,7 @@
#include <unistd.h>
#include "access/xlog.h"
+#include "bestatus.h"
#include "catalog/catalog.h"
#include "catalog/storage.h"
#include "executor/instrument.h"
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index c2c445dbf4..0bb2132c71 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -41,9 +41,9 @@
#include "postgres.h"
+#include "bestatus.h"
#include "executor/instrument.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/fd.h"
#include "storage/buffile.h"
#include "storage/buf_internals.h"
diff --git a/src/backend/storage/file/copydir.c b/src/backend/storage/file/copydir.c
index 1f766d20d1..a0401ee494 100644
--- a/src/backend/storage/file/copydir.c
+++ b/src/backend/storage/file/copydir.c
@@ -22,10 +22,10 @@
#include <unistd.h>
#include <sys/stat.h>
+#include "bestatus.h"
#include "storage/copydir.h"
#include "storage/fd.h"
#include "miscadmin.h"
-#include "pgstat.h"
/*
* copydir: copy a directory
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index 213de7698a..6bc5fd6089 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -82,6 +82,7 @@
#include "miscadmin.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "catalog/pg_tablespace.h"
#include "common/file_perm.h"
#include "pgstat.h"
diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c
index cab7ae74ca..c7c248878a 100644
--- a/src/backend/storage/ipc/dsm.c
+++ b/src/backend/storage/ipc/dsm.c
@@ -197,6 +197,15 @@ dsm_postmaster_startup(PGShmemHeader *shim)
dsm_control->maxitems = maxitems;
}
+/*
+ * clear dsm_init state on child start.
+ */
+void
+dsm_child_init(void)
+{
+ dsm_init_done = false;
+}
+
/*
* Determine whether the control segment from the previous postmaster
* invocation still exists. If so, remove the dynamic shared memory
@@ -423,6 +432,15 @@ dsm_set_control_handle(dsm_handle h)
}
#endif
+/*
+ * Return if dsm feature is available on this process.
+ */
+bool
+dsm_is_available(void)
+{
+ return dsm_control != NULL;
+}
+
/*
* Create a new dynamic shared memory segment.
*
@@ -440,8 +458,7 @@ dsm_create(Size size, int flags)
uint32 i;
uint32 nitems;
- /* Unsafe in postmaster (and pointless in a stand-alone backend). */
- Assert(IsUnderPostmaster);
+ Assert(dsm_is_available());
if (!dsm_init_done)
dsm_backend_startup();
@@ -537,8 +554,7 @@ dsm_attach(dsm_handle h)
uint32 i;
uint32 nitems;
- /* Unsafe in postmaster (and pointless in a stand-alone backend). */
- Assert(IsUnderPostmaster);
+ Assert(dsm_is_available());
if (!dsm_init_done)
dsm_backend_startup();
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index aeda32c9c5..e84275d4c2 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -61,8 +61,8 @@
#ifdef HAVE_SYS_SHM_H
#include <sys/shm.h>
#endif
+#include "bestatus.h"
#include "common/file_perm.h"
-#include "pgstat.h"
#include "portability/mem.h"
#include "storage/dsm_impl.h"
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 5965d3620f..97bca9be24 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -150,6 +150,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -281,8 +282,13 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
/* Initialize dynamic shared memory facilities. */
if (!IsUnderPostmaster)
+ {
dsm_postmaster_startup(shim);
+ /* Stats collector uses dynamic shared memory */
+ StatsShmemInit();
+ }
+
/*
* Now give loadable modules a chance to set up their shmem allocations
*/
diff --git a/src/backend/storage/ipc/latch.c b/src/backend/storage/ipc/latch.c
index 7da337d11f..97526f1c72 100644
--- a/src/backend/storage/ipc/latch.c
+++ b/src/backend/storage/ipc/latch.c
@@ -43,8 +43,8 @@
#include <poll.h>
#endif
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "port/atomics.h"
#include "portability/instr_time.h"
#include "postmaster/postmaster.h"
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index cf93357997..e893984383 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -51,9 +51,9 @@
#include "access/twophase.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "catalog/catalog.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/proc.h"
#include "storage/procarray.h"
#include "storage/spin.h"
diff --git a/src/backend/storage/ipc/shm_mq.c b/src/backend/storage/ipc/shm_mq.c
index 6e471c3e43..cfa5c9089f 100644
--- a/src/backend/storage/ipc/shm_mq.c
+++ b/src/backend/storage/ipc/shm_mq.c
@@ -18,8 +18,8 @@
#include "postgres.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "storage/procsignal.h"
#include "storage/shm_mq.h"
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index 4d10e57a80..243da57c49 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -21,8 +21,8 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "access/xloginsert.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/bufmgr.h"
#include "storage/lmgr.h"
#include "storage/proc.h"
diff --git a/src/backend/storage/lmgr/deadlock.c b/src/backend/storage/lmgr/deadlock.c
index 74eb449060..dd76088a29 100644
--- a/src/backend/storage/lmgr/deadlock.c
+++ b/src/backend/storage/lmgr/deadlock.c
@@ -25,6 +25,7 @@
*/
#include "postgres.h"
+#include "bestatus.h"
#include "miscadmin.h"
#include "pg_trace.h"
#include "pgstat.h"
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 81dac45ae5..2cd4d5531e 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -76,8 +76,8 @@
*/
#include "postgres.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "pg_trace.h"
#include "postmaster/postmaster.h"
#include "replication/slot.h"
@@ -521,6 +521,9 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_TBM, "tbm");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
+ LWLockRegisterTranche(LWTRANCHE_STATS_DSA, "stats table dsa");
+ LWLockRegisterTranche(LWTRANCHE_STATS_DB, "db stats");
+ LWLockRegisterTranche(LWTRANCHE_STATS_FUNC_TABLE, "table/func stats");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index db47843229..97eccb35d3 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -49,3 +49,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
LogicalRepWorkerLock 43
CLogTruncationLock 44
+StatsLock 45
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index 6fc11f26f0..a8efa7cc5f 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -194,8 +194,8 @@
#include "access/twophase_rmgr.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/bufmgr.h"
#include "storage/predicate.h"
#include "storage/predicate_internals.h"
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 0da5b19719..a60fd02894 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -38,8 +38,8 @@
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
+#include "bestatus.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "replication/slot.h"
#include "replication/syncrep.h"
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index 2aba2dfe91..9e9995ae50 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -28,7 +28,7 @@
#include "miscadmin.h"
#include "access/xlogutils.h"
#include "access/xlog.h"
-#include "pgstat.h"
+#include "bestatus.h"
#include "portability/instr_time.h"
#include "postmaster/bgwriter.h"
#include "storage/fd.h"
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cfd507b2..eeda0c04f5 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -39,6 +39,7 @@
#include "access/parallel.h"
#include "access/printtup.h"
#include "access/xact.h"
+#include "bestatus.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/prepare.h"
@@ -3159,6 +3160,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_update_stat(true);
+ }
}
@@ -3733,6 +3740,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4173,9 +4181,17 @@ PostgresMain(int argc, char *argv[],
}
else
{
- ProcessCompletedNotifies();
- pgstat_report_stat(false);
+ long stats_timeout;
+ ProcessCompletedNotifies();
+
+ stats_timeout = pgstat_update_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle", false);
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4210,7 +4226,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4218,6 +4234,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index d330a88e3c..c0975a8259 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -21,6 +21,7 @@
#include "access/sysattr.h"
#include "access/table.h"
+#include "bestatus.h"
#include "catalog/catalog.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_type.h"
@@ -29,7 +30,6 @@
#include "common/keywords.h"
#include "funcapi.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "parser/scansup.h"
#include "postmaster/syslogger.h"
#include "rewrite/rewriteHandler.h"
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b6ba856ebe..667e8e5560 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "bestatus.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_type.h"
#include "common/ip.h"
@@ -33,7 +34,7 @@
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
+extern PgStat_BgWriter bgwriterStats;
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
@@ -1193,7 +1194,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_xact_commit);
@@ -1209,7 +1210,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_xact_rollback);
@@ -1225,7 +1226,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_blocks_fetched);
@@ -1241,7 +1242,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_blocks_hit);
@@ -1257,7 +1258,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_returned);
@@ -1273,7 +1274,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_fetched);
@@ -1289,7 +1290,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_inserted);
@@ -1305,7 +1306,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_updated);
@@ -1321,7 +1322,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_deleted);
@@ -1336,7 +1337,7 @@ pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS)
TimestampTz result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = dbentry->stat_reset_timestamp;
@@ -1354,7 +1355,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = dbentry->n_temp_files;
@@ -1370,7 +1371,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = dbentry->n_temp_bytes;
@@ -1385,7 +1386,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_tablespace);
@@ -1400,7 +1401,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_lock);
@@ -1415,7 +1416,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_snapshot);
@@ -1430,7 +1431,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_bufferpin);
@@ -1445,7 +1446,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_startup_deadlock);
@@ -1460,7 +1461,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (
@@ -1480,7 +1481,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_deadlocks);
@@ -1496,7 +1497,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
PgStat_StatDBEntry *dbentry;
/* convert counter from microsec to millisec for display */
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = ((double) dbentry->n_block_read_time) / 1000.0;
@@ -1512,7 +1513,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
PgStat_StatDBEntry *dbentry;
/* convert counter from microsec to millisec for display */
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = ((double) dbentry->n_block_write_time) / 1000.0;
@@ -1867,6 +1868,9 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
/* Get statistics about the archiver process */
archiver_stats = pgstat_fetch_stat_archiver();
+ if (archiver_stats == NULL)
+ PG_RETURN_NULL();
+
/* Fill values and NULLs */
values[0] = Int64GetDatum(archiver_stats->archived_count);
if (*(archiver_stats->last_archived_wal) == '\0')
@@ -1896,6 +1900,5 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(
- heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c
index 5e61d908fd..2dd99f935d 100644
--- a/src/backend/utils/cache/relmapper.c
+++ b/src/backend/utils/cache/relmapper.c
@@ -46,11 +46,11 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "access/xloginsert.h"
+#include "bestatus.h"
#include "catalog/catalog.h"
#include "catalog/pg_tablespace.h"
#include "catalog/storage.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "storage/fd.h"
#include "storage/lwlock.h"
#include "utils/inval.h"
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index fd51934aaf..994351ac2d 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -32,6 +32,7 @@ volatile sig_atomic_t QueryCancelPending = false;
volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile sig_atomic_t ConfigReloadPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index bd2e4e89d8..1eabc0f41d 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -31,12 +31,12 @@
#endif
#include "access/htup_details.h"
+#include "bestatus.h"
#include "catalog/pg_authid.h"
#include "common/file_perm.h"
#include "libpq/libpq.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/postmaster.h"
#include "storage/fd.h"
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index a5ee209f91..e5dca7fe03 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -26,6 +26,7 @@
#include "access/sysattr.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "bestatus.h"
#include "catalog/catalog.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
@@ -72,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -628,6 +630,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -685,7 +689,10 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
/* Initialize stats collection --- must happen before first xact */
if (!bootstrap)
+ {
+ pgstat_bearray_initialize();
pgstat_initialize();
+ }
/*
* Load relcache entries for the shared system catalogs. This must create
@@ -1238,6 +1245,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 41d477165c..fb7856517e 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/twophase.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
+#include "bestatus.h"
#include "catalog/namespace.h"
#include "catalog/pg_authid.h"
#include "commands/async.h"
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3e1c3863c4..25b3b2a079 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/bestatus.h b/src/include/bestatus.h
new file mode 100644
index 0000000000..b7f6a93130
--- /dev/null
+++ b/src/include/bestatus.h
@@ -0,0 +1,555 @@
+/* ----------
+ * bestatus.h
+ *
+ * Definitions for the PostgreSQL backend status monitor facility
+ *
+ * Copyright (c) 2001-2018, PostgreSQL Global Development Group
+ *
+ * src/include/bestatus.h
+ * ----------
+ */
+#ifndef BESTATUS_H
+#define BESTATUS_H
+
+#include "datatype/timestamp.h"
+#include "libpq/pqcomm.h"
+#include "storage/proc.h"
+
+/* ----------
+ * Backend types
+ * ----------
+ */
+typedef enum BackendType
+{
+ B_AUTOVAC_LAUNCHER,
+ B_AUTOVAC_WORKER,
+ B_BACKEND,
+ B_BG_WORKER,
+ B_BG_WRITER,
+ B_CHECKPOINTER,
+ B_STARTUP,
+ B_WAL_RECEIVER,
+ B_WAL_SENDER,
+ B_WAL_WRITER,
+ B_ARCHIVER
+} BackendType;
+
+
+/* ----------
+ * Backend states
+ * ----------
+ */
+typedef enum BackendState
+{
+ STATE_UNDEFINED,
+ STATE_IDLE,
+ STATE_RUNNING,
+ STATE_IDLEINTRANSACTION,
+ STATE_FASTPATH,
+ STATE_IDLEINTRANSACTION_ABORTED,
+ STATE_DISABLED
+} BackendState;
+
+
+/* ----------
+ * Wait Classes
+ * ----------
+ */
+#define PG_WAIT_LWLOCK 0x01000000U
+#define PG_WAIT_LOCK 0x03000000U
+#define PG_WAIT_BUFFER_PIN 0x04000000U
+#define PG_WAIT_ACTIVITY 0x05000000U
+#define PG_WAIT_CLIENT 0x06000000U
+#define PG_WAIT_EXTENSION 0x07000000U
+#define PG_WAIT_IPC 0x08000000U
+#define PG_WAIT_TIMEOUT 0x09000000U
+#define PG_WAIT_IO 0x0A000000U
+
+/* ----------
+ * Wait Events - Activity
+ *
+ * Use this category when a process is waiting because it has no work to do,
+ * unless the "Client" or "Timeout" category describes the situation better.
+ * Typically, this should only be used for background processes.
+ * ----------
+ */
+typedef enum
+{
+ WAIT_EVENT_ARCHIVER_MAIN = PG_WAIT_ACTIVITY,
+ WAIT_EVENT_AUTOVACUUM_MAIN,
+ WAIT_EVENT_BGWRITER_HIBERNATE,
+ WAIT_EVENT_BGWRITER_MAIN,
+ WAIT_EVENT_CHECKPOINTER_MAIN,
+ WAIT_EVENT_LOGICAL_APPLY_MAIN,
+ WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
+ WAIT_EVENT_RECOVERY_WAL_ALL,
+ WAIT_EVENT_RECOVERY_WAL_STREAM,
+ WAIT_EVENT_SYSLOGGER_MAIN,
+ WAIT_EVENT_WAL_RECEIVER_MAIN,
+ WAIT_EVENT_WAL_SENDER_MAIN,
+ WAIT_EVENT_WAL_WRITER_MAIN
+} WaitEventActivity;
+
+/* ----------
+ * Wait Events - Client
+ *
+ * Use this category when a process is waiting to send data to or receive data
+ * from the frontend process to which it is connected. This is never used for
+ * a background process, which has no client connection.
+ * ----------
+ */
+typedef enum
+{
+ WAIT_EVENT_CLIENT_READ = PG_WAIT_CLIENT,
+ WAIT_EVENT_CLIENT_WRITE,
+ WAIT_EVENT_LIBPQWALRECEIVER_CONNECT,
+ WAIT_EVENT_LIBPQWALRECEIVER_RECEIVE,
+ WAIT_EVENT_SSL_OPEN_SERVER,
+ WAIT_EVENT_WAL_RECEIVER_WAIT_START,
+ WAIT_EVENT_WAL_SENDER_WAIT_WAL,
+ WAIT_EVENT_WAL_SENDER_WRITE_DATA
+} WaitEventClient;
+
+/* ----------
+ * Wait Events - IPC
+ *
+ * Use this category when a process cannot complete the work it is doing because
+ * it is waiting for a notification from another process.
+ * ----------
+ */
+typedef enum
+{
+ WAIT_EVENT_BGWORKER_SHUTDOWN = PG_WAIT_IPC,
+ WAIT_EVENT_BGWORKER_STARTUP,
+ WAIT_EVENT_BTREE_PAGE,
+ WAIT_EVENT_CLOG_GROUP_UPDATE,
+ WAIT_EVENT_EXECUTE_GATHER,
+ WAIT_EVENT_HASH_BATCH_ALLOCATING,
+ WAIT_EVENT_HASH_BATCH_ELECTING,
+ WAIT_EVENT_HASH_BATCH_LOADING,
+ WAIT_EVENT_HASH_BUILD_ALLOCATING,
+ WAIT_EVENT_HASH_BUILD_ELECTING,
+ WAIT_EVENT_HASH_BUILD_HASHING_INNER,
+ WAIT_EVENT_HASH_BUILD_HASHING_OUTER,
+ WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING,
+ WAIT_EVENT_HASH_GROW_BATCHES_DECIDING,
+ WAIT_EVENT_HASH_GROW_BATCHES_ELECTING,
+ WAIT_EVENT_HASH_GROW_BATCHES_FINISHING,
+ WAIT_EVENT_HASH_GROW_BATCHES_REPARTITIONING,
+ WAIT_EVENT_HASH_GROW_BUCKETS_ALLOCATING,
+ WAIT_EVENT_HASH_GROW_BUCKETS_ELECTING,
+ WAIT_EVENT_HASH_GROW_BUCKETS_REINSERTING,
+ WAIT_EVENT_LOGICAL_SYNC_DATA,
+ WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE,
+ WAIT_EVENT_MQ_INTERNAL,
+ WAIT_EVENT_MQ_PUT_MESSAGE,
+ WAIT_EVENT_MQ_RECEIVE,
+ WAIT_EVENT_MQ_SEND,
+ WAIT_EVENT_PARALLEL_BITMAP_SCAN,
+ WAIT_EVENT_PARALLEL_CREATE_INDEX_SCAN,
+ WAIT_EVENT_PARALLEL_FINISH,
+ WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
+ WAIT_EVENT_PROMOTE,
+ WAIT_EVENT_REPLICATION_ORIGIN_DROP,
+ WAIT_EVENT_REPLICATION_SLOT_DROP,
+ WAIT_EVENT_SAFE_SNAPSHOT,
+ WAIT_EVENT_SYNC_REP
+} WaitEventIPC;
+
+/* ----------
+ * Wait Events - Timeout
+ *
+ * Use this category when a process is waiting for a timeout to expire.
+ * ----------
+ */
+typedef enum
+{
+ WAIT_EVENT_BASE_BACKUP_THROTTLE = PG_WAIT_TIMEOUT,
+ WAIT_EVENT_PG_SLEEP,
+ WAIT_EVENT_RECOVERY_APPLY_DELAY
+} WaitEventTimeout;
+
+/* ----------
+ * Wait Events - IO
+ *
+ * Use this category when a process is waiting for a IO.
+ * ----------
+ */
+typedef enum
+{
+ WAIT_EVENT_BUFFILE_READ = PG_WAIT_IO,
+ WAIT_EVENT_BUFFILE_WRITE,
+ WAIT_EVENT_CONTROL_FILE_READ,
+ WAIT_EVENT_CONTROL_FILE_SYNC,
+ WAIT_EVENT_CONTROL_FILE_SYNC_UPDATE,
+ WAIT_EVENT_CONTROL_FILE_WRITE,
+ WAIT_EVENT_CONTROL_FILE_WRITE_UPDATE,
+ WAIT_EVENT_COPY_FILE_READ,
+ WAIT_EVENT_COPY_FILE_WRITE,
+ WAIT_EVENT_DATA_FILE_EXTEND,
+ WAIT_EVENT_DATA_FILE_FLUSH,
+ WAIT_EVENT_DATA_FILE_IMMEDIATE_SYNC,
+ WAIT_EVENT_DATA_FILE_PREFETCH,
+ WAIT_EVENT_DATA_FILE_READ,
+ WAIT_EVENT_DATA_FILE_SYNC,
+ WAIT_EVENT_DATA_FILE_TRUNCATE,
+ WAIT_EVENT_DATA_FILE_WRITE,
+ WAIT_EVENT_DSM_FILL_ZERO_WRITE,
+ WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ,
+ WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC,
+ WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE,
+ WAIT_EVENT_LOCK_FILE_CREATE_READ,
+ WAIT_EVENT_LOCK_FILE_CREATE_SYNC,
+ WAIT_EVENT_LOCK_FILE_CREATE_WRITE,
+ WAIT_EVENT_LOCK_FILE_RECHECKDATADIR_READ,
+ WAIT_EVENT_LOGICAL_REWRITE_CHECKPOINT_SYNC,
+ WAIT_EVENT_LOGICAL_REWRITE_MAPPING_SYNC,
+ WAIT_EVENT_LOGICAL_REWRITE_MAPPING_WRITE,
+ WAIT_EVENT_LOGICAL_REWRITE_SYNC,
+ WAIT_EVENT_LOGICAL_REWRITE_TRUNCATE,
+ WAIT_EVENT_LOGICAL_REWRITE_WRITE,
+ WAIT_EVENT_RELATION_MAP_READ,
+ WAIT_EVENT_RELATION_MAP_SYNC,
+ WAIT_EVENT_RELATION_MAP_WRITE,
+ WAIT_EVENT_REORDER_BUFFER_READ,
+ WAIT_EVENT_REORDER_BUFFER_WRITE,
+ WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ,
+ WAIT_EVENT_REPLICATION_SLOT_READ,
+ WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
+ WAIT_EVENT_REPLICATION_SLOT_SYNC,
+ WAIT_EVENT_REPLICATION_SLOT_WRITE,
+ WAIT_EVENT_SLRU_FLUSH_SYNC,
+ WAIT_EVENT_SLRU_READ,
+ WAIT_EVENT_SLRU_SYNC,
+ WAIT_EVENT_SLRU_WRITE,
+ WAIT_EVENT_SNAPBUILD_READ,
+ WAIT_EVENT_SNAPBUILD_SYNC,
+ WAIT_EVENT_SNAPBUILD_WRITE,
+ WAIT_EVENT_TIMELINE_HISTORY_FILE_SYNC,
+ WAIT_EVENT_TIMELINE_HISTORY_FILE_WRITE,
+ WAIT_EVENT_TIMELINE_HISTORY_READ,
+ WAIT_EVENT_TIMELINE_HISTORY_SYNC,
+ WAIT_EVENT_TIMELINE_HISTORY_WRITE,
+ WAIT_EVENT_TWOPHASE_FILE_READ,
+ WAIT_EVENT_TWOPHASE_FILE_SYNC,
+ WAIT_EVENT_TWOPHASE_FILE_WRITE,
+ WAIT_EVENT_WALSENDER_TIMELINE_HISTORY_READ,
+ WAIT_EVENT_WAL_BOOTSTRAP_SYNC,
+ WAIT_EVENT_WAL_BOOTSTRAP_WRITE,
+ WAIT_EVENT_WAL_COPY_READ,
+ WAIT_EVENT_WAL_COPY_SYNC,
+ WAIT_EVENT_WAL_COPY_WRITE,
+ WAIT_EVENT_WAL_INIT_SYNC,
+ WAIT_EVENT_WAL_INIT_WRITE,
+ WAIT_EVENT_WAL_READ,
+ WAIT_EVENT_WAL_SYNC,
+ WAIT_EVENT_WAL_SYNC_METHOD_ASSIGN,
+ WAIT_EVENT_WAL_WRITE
+} WaitEventIO;
+
+/* ----------
+ * Command type for progress reporting purposes
+ * ----------
+ */
+typedef enum ProgressCommandType
+{
+ PROGRESS_COMMAND_INVALID,
+ PROGRESS_COMMAND_VACUUM
+} ProgressCommandType;
+
+#define PGSTAT_NUM_PROGRESS_PARAM 10
+
+/* ----------
+ * Shared-memory data structures
+ * ----------
+ */
+
+
+/*
+ * PgBackendSSLStatus
+ *
+ * For each backend, we keep the SSL status in a separate struct, that
+ * is only filled in if SSL is enabled.
+ *
+ * All char arrays must be null-terminated.
+ */
+typedef struct PgBackendSSLStatus
+{
+ /* Information about SSL connection */
+ int ssl_bits;
+ bool ssl_compression;
+ char ssl_version[NAMEDATALEN];
+ char ssl_cipher[NAMEDATALEN];
+ char ssl_client_dn[NAMEDATALEN];
+
+ /*
+ * serial number is max "20 octets" per RFC 5280, so this size should be
+ * fine
+ */
+ char ssl_client_serial[NAMEDATALEN];
+
+ char ssl_issuer_dn[NAMEDATALEN];
+} PgBackendSSLStatus;
+
+
+/* ----------
+ * PgBackendStatus
+ *
+ * Each live backend maintains a PgBackendStatus struct in shared memory
+ * showing its current activity. (The structs are allocated according to
+ * BackendId, but that is not critical.) Note that the collector process
+ * has no involvement in, or even access to, these structs.
+ *
+ * Each auxiliary process also maintains a PgBackendStatus struct in shared
+ * memory.
+ * ----------
+ */
+typedef struct PgBackendStatus
+{
+ /*
+ * To avoid locking overhead, we use the following protocol: a backend
+ * increments st_changecount before modifying its entry, and again after
+ * finishing a modification. A would-be reader should note the value of
+ * st_changecount, copy the entry into private memory, then check
+ * st_changecount again. If the value hasn't changed, and if it's even,
+ * the copy is valid; otherwise start over. This makes updates cheap
+ * while reads are potentially expensive, but that's the tradeoff we want.
+ *
+ * The above protocol needs the memory barriers to ensure that the
+ * apparent order of execution is as it desires. Otherwise, for example,
+ * the CPU might rearrange the code so that st_changecount is incremented
+ * twice before the modification on a machine with weak memory ordering.
+ * This surprising result can lead to bugs.
+ */
+ int st_changecount;
+
+ /* The entry is valid iff st_procpid > 0, unused if st_procpid == 0 */
+ int st_procpid;
+
+ /* Type of backends */
+ BackendType st_backendType;
+
+ /* Times when current backend, transaction, and activity started */
+ TimestampTz st_proc_start_timestamp;
+ TimestampTz st_xact_start_timestamp;
+ TimestampTz st_activity_start_timestamp;
+ TimestampTz st_state_start_timestamp;
+
+ /* Database OID, owning user's OID, connection client address */
+ Oid st_databaseid;
+ Oid st_userid;
+ SockAddr st_clientaddr;
+ char *st_clienthostname; /* MUST be null-terminated */
+
+ /* Information about SSL connection */
+ bool st_ssl;
+ PgBackendSSLStatus *st_sslstatus;
+
+ /* current state */
+ BackendState st_state;
+
+ /* application name; MUST be null-terminated */
+ char *st_appname;
+
+ /*
+ * Current command string; MUST be null-terminated. Note that this string
+ * possibly is truncated in the middle of a multi-byte character. As
+ * activity strings are stored more frequently than read, that allows to
+ * move the cost of correct truncation to the display side. Use
+ * pgstat_clip_activity() to truncate correctly.
+ */
+ char *st_activity_raw;
+
+ /*
+ * Command progress reporting. Any command which wishes can advertise
+ * that it is running by setting st_progress_command,
+ * st_progress_command_target, and st_progress_param[].
+ * st_progress_command_target should be the OID of the relation which the
+ * command targets (we assume there's just one, as this is meant for
+ * utility commands), but the meaning of each element in the
+ * st_progress_param array is command-specific.
+ */
+ ProgressCommandType st_progress_command;
+ Oid st_progress_command_target;
+ int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+} PgBackendStatus;
+
+/*
+ * Macros to load and store st_changecount with the memory barriers.
+ *
+ * pgstat_increment_changecount_before() and
+ * pgstat_increment_changecount_after() need to be called before and after
+ * PgBackendStatus entries are modified, respectively. This makes sure that
+ * st_changecount is incremented around the modification.
+ *
+ * Also pgstat_save_changecount_before() and pgstat_save_changecount_after()
+ * need to be called before and after PgBackendStatus entries are copied into
+ * private memory, respectively.
+ */
+#define pgstat_increment_changecount_before(beentry) \
+ do { \
+ beentry->st_changecount++; \
+ pg_write_barrier(); \
+ } while (0)
+
+#define pgstat_increment_changecount_after(beentry) \
+ do { \
+ pg_write_barrier(); \
+ beentry->st_changecount++; \
+ Assert((beentry->st_changecount & 1) == 0); \
+ } while (0)
+
+#define pgstat_save_changecount_before(beentry, save_changecount) \
+ do { \
+ save_changecount = beentry->st_changecount; \
+ pg_read_barrier(); \
+ } while (0)
+
+#define pgstat_save_changecount_after(beentry, save_changecount) \
+ do { \
+ pg_read_barrier(); \
+ save_changecount = beentry->st_changecount; \
+ } while (0)
+
+/* ----------
+ * LocalPgBackendStatus
+ *
+ * When we build the backend status array, we use LocalPgBackendStatus to be
+ * able to add new values to the struct when needed without adding new fields
+ * to the shared memory. It contains the backend status as a first member.
+ * ----------
+ */
+typedef struct LocalPgBackendStatus
+{
+ /*
+ * Local version of the backend status entry.
+ */
+ PgBackendStatus backendStatus;
+
+ /*
+ * The xid of the current transaction if available, InvalidTransactionId
+ * if not.
+ */
+ TransactionId backend_xid;
+
+ /*
+ * The xmin of the current session if available, InvalidTransactionId if
+ * not.
+ */
+ TransactionId backend_xmin;
+} LocalPgBackendStatus;
+
+/* ----------
+ * GUC parameters
+ * ----------
+ */
+extern bool pgstat_track_activities;
+extern PGDLLIMPORT int pgstat_track_activity_query_size;
+
+/* ----------
+ * Functions called from backends
+ * ----------
+ */
+extern void pgstat_bearray_initialize(void);
+extern void pgstat_bestart(void);
+
+extern const char *pgstat_get_wait_event(uint32 wait_event_info);
+extern const char *pgstat_get_wait_event_type(uint32 wait_event_info);
+extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
+extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
+ int buflen);
+extern const char *pgstat_get_backend_desc(BackendType backendType);
+
+extern void pgstat_progress_start_command(ProgressCommandType cmdtype,
+ Oid relid);
+extern void pgstat_progress_update_param(int index, int64 val);
+extern void pgstat_progress_update_multi_param(int nparam, const int *index,
+ const int64 *val);
+extern void pgstat_progress_end_command(void);
+
+extern char *pgstat_clip_activity(const char *raw_activity);
+
+extern void AtEOXact_BEStatus(bool isCommit);
+extern void AtPrepare_BEStatus(void);
+/* ----------
+ * pgstat_report_wait_start() -
+ *
+ * Called from places where server process needs to wait. This is called
+ * to report wait event information. The wait information is stored
+ * as 4-bytes where first byte represents the wait event class (type of
+ * wait, for different types of wait, refer WaitClass) and the next
+ * 3-bytes represent the actual wait event. Currently 2-bytes are used
+ * for wait event which is sufficient for current usage, 1-byte is
+ * reserved for future usage.
+ *
+ * NB: this *must* be able to survive being called before MyProc has been
+ * initialized.
+ * ----------
+ */
+static inline void
+pgstat_report_wait_start(uint32 wait_event_info)
+{
+ volatile PGPROC *proc = MyProc;
+
+ if (!pgstat_track_activities || !proc)
+ return;
+
+ /*
+ * Since this is a four-byte field which is always read and written as
+ * four-bytes, updates are atomic.
+ */
+ proc->wait_event_info = wait_event_info;
+}
+
+/* ----------
+ * pgstat_report_wait_end() -
+ *
+ * Called to report end of a wait.
+ *
+ * NB: this *must* be able to survive being called before MyProc has been
+ * initialized.
+ * ----------
+ */
+static inline void
+pgstat_report_wait_end(void)
+{
+ volatile PGPROC *proc = MyProc;
+
+ if (!pgstat_track_activities || !proc)
+ return;
+
+ /*
+ * Since this is a four-byte field which is always read and written as
+ * four-bytes, updates are atomic.
+ */
+ proc->wait_event_info = 0;
+}
+extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
+extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
+extern int pgstat_fetch_stat_numbackends(void);
+
+/* For shared memory allocation/initialize */
+extern Size BackendStatusShmemSize(void);
+extern void CreateSharedBackendStatus(void);
+
+void pgstat_report_xact_timestamp(TimestampTz tstamp);
+void pgstat_bestat_initialize(void);
+
+extern void pgstat_report_activity(BackendState state, const char *cmd_str);
+extern void pgstat_report_appname(const char *appname);
+extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
+extern const char *pgstat_get_wait_event(uint32 wait_event_info);
+extern const char *pgstat_get_wait_event_type(uint32 wait_event_info);
+extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
+extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
+ int buflen);
+extern const char *pgstat_get_backend_desc(BackendType backendType);
+
+extern void pgstat_progress_start_command(ProgressCommandType cmdtype,
+ Oid relid);
+extern void pgstat_progress_update_param(int index, int64 val);
+extern void pgstat_progress_update_multi_param(int nparam, const int *index,
+ const int64 *val);
+extern void pgstat_progress_end_command(void);
+
+#endif /* BESTATUS_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 63a7653457..49131a6d5b 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -82,6 +82,7 @@ extern PGDLLIMPORT volatile sig_atomic_t InterruptPending;
extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -403,7 +404,6 @@ typedef enum
CheckpointerProcess,
WalWriterProcess,
WalReceiverProcess,
-
NUM_AUXPROCTYPES /* Must be last! */
} AuxProcType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 471877d2df..92c9adf48e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL statistics collector facility.
*
* Copyright (c) 2001-2019, PostgreSQL Global Development Group
*
@@ -13,11 +13,9 @@
#include "datatype/timestamp.h"
#include "fmgr.h"
-#include "libpq/pqcomm.h"
-#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
-#include "storage/proc.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,32 +39,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -115,13 +87,6 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_blocks_hit;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -180,236 +145,12 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
/* ----------
- * PgStat_MsgHdr The common message header
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgHdr
+typedef struct PgStat_BgWriter
{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
PgStat_Counter m_timed_checkpoints;
PgStat_Counter m_requested_checkpoints;
PgStat_Counter m_buf_written_checkpoints;
@@ -420,40 +161,16 @@ typedef struct PgStat_MsgBgWriter
PgStat_Counter m_buf_alloc;
PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to apply.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when applying to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -485,81 +202,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Statistic collector data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -601,10 +245,13 @@ typedef struct PgStat_StatDBEntry
/*
* tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * the handles and pointers out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables;
+ dshash_table_handle functions;
+ /* for snapshot tables */
+ HTAB *snapshot_tables;
+ HTAB *snapshot_functions;
} PgStat_StatDBEntry;
@@ -660,7 +307,7 @@ typedef struct PgStat_StatFuncEntry
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -676,7 +323,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -694,432 +341,6 @@ typedef struct PgStat_GlobalStats
TimestampTz stat_reset_timestamp;
} PgStat_GlobalStats;
-
-/* ----------
- * Backend types
- * ----------
- */
-typedef enum BackendType
-{
- B_AUTOVAC_LAUNCHER,
- B_AUTOVAC_WORKER,
- B_BACKEND,
- B_BG_WORKER,
- B_BG_WRITER,
- B_ARCHIVER,
- B_CHECKPOINTER,
- B_STARTUP,
- B_WAL_RECEIVER,
- B_WAL_SENDER,
- B_WAL_WRITER
-} BackendType;
-
-
-/* ----------
- * Backend states
- * ----------
- */
-typedef enum BackendState
-{
- STATE_UNDEFINED,
- STATE_IDLE,
- STATE_RUNNING,
- STATE_IDLEINTRANSACTION,
- STATE_FASTPATH,
- STATE_IDLEINTRANSACTION_ABORTED,
- STATE_DISABLED
-} BackendState;
-
-
-/* ----------
- * Wait Classes
- * ----------
- */
-#define PG_WAIT_LWLOCK 0x01000000U
-#define PG_WAIT_LOCK 0x03000000U
-#define PG_WAIT_BUFFER_PIN 0x04000000U
-#define PG_WAIT_ACTIVITY 0x05000000U
-#define PG_WAIT_CLIENT 0x06000000U
-#define PG_WAIT_EXTENSION 0x07000000U
-#define PG_WAIT_IPC 0x08000000U
-#define PG_WAIT_TIMEOUT 0x09000000U
-#define PG_WAIT_IO 0x0A000000U
-
-/* ----------
- * Wait Events - Activity
- *
- * Use this category when a process is waiting because it has no work to do,
- * unless the "Client" or "Timeout" category describes the situation better.
- * Typically, this should only be used for background processes.
- * ----------
- */
-typedef enum
-{
- WAIT_EVENT_ARCHIVER_MAIN = PG_WAIT_ACTIVITY,
- WAIT_EVENT_AUTOVACUUM_MAIN,
- WAIT_EVENT_BGWRITER_HIBERNATE,
- WAIT_EVENT_BGWRITER_MAIN,
- WAIT_EVENT_CHECKPOINTER_MAIN,
- WAIT_EVENT_LOGICAL_APPLY_MAIN,
- WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
- WAIT_EVENT_RECOVERY_WAL_ALL,
- WAIT_EVENT_RECOVERY_WAL_STREAM,
- WAIT_EVENT_SYSLOGGER_MAIN,
- WAIT_EVENT_WAL_RECEIVER_MAIN,
- WAIT_EVENT_WAL_SENDER_MAIN,
- WAIT_EVENT_WAL_WRITER_MAIN
-} WaitEventActivity;
-
-/* ----------
- * Wait Events - Client
- *
- * Use this category when a process is waiting to send data to or receive data
- * from the frontend process to which it is connected. This is never used for
- * a background process, which has no client connection.
- * ----------
- */
-typedef enum
-{
- WAIT_EVENT_CLIENT_READ = PG_WAIT_CLIENT,
- WAIT_EVENT_CLIENT_WRITE,
- WAIT_EVENT_LIBPQWALRECEIVER_CONNECT,
- WAIT_EVENT_LIBPQWALRECEIVER_RECEIVE,
- WAIT_EVENT_SSL_OPEN_SERVER,
- WAIT_EVENT_WAL_RECEIVER_WAIT_START,
- WAIT_EVENT_WAL_SENDER_WAIT_WAL,
- WAIT_EVENT_WAL_SENDER_WRITE_DATA
-} WaitEventClient;
-
-/* ----------
- * Wait Events - IPC
- *
- * Use this category when a process cannot complete the work it is doing because
- * it is waiting for a notification from another process.
- * ----------
- */
-typedef enum
-{
- WAIT_EVENT_BGWORKER_SHUTDOWN = PG_WAIT_IPC,
- WAIT_EVENT_BGWORKER_STARTUP,
- WAIT_EVENT_BTREE_PAGE,
- WAIT_EVENT_CLOG_GROUP_UPDATE,
- WAIT_EVENT_EXECUTE_GATHER,
- WAIT_EVENT_HASH_BATCH_ALLOCATING,
- WAIT_EVENT_HASH_BATCH_ELECTING,
- WAIT_EVENT_HASH_BATCH_LOADING,
- WAIT_EVENT_HASH_BUILD_ALLOCATING,
- WAIT_EVENT_HASH_BUILD_ELECTING,
- WAIT_EVENT_HASH_BUILD_HASHING_INNER,
- WAIT_EVENT_HASH_BUILD_HASHING_OUTER,
- WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING,
- WAIT_EVENT_HASH_GROW_BATCHES_DECIDING,
- WAIT_EVENT_HASH_GROW_BATCHES_ELECTING,
- WAIT_EVENT_HASH_GROW_BATCHES_FINISHING,
- WAIT_EVENT_HASH_GROW_BATCHES_REPARTITIONING,
- WAIT_EVENT_HASH_GROW_BUCKETS_ALLOCATING,
- WAIT_EVENT_HASH_GROW_BUCKETS_ELECTING,
- WAIT_EVENT_HASH_GROW_BUCKETS_REINSERTING,
- WAIT_EVENT_LOGICAL_SYNC_DATA,
- WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE,
- WAIT_EVENT_MQ_INTERNAL,
- WAIT_EVENT_MQ_PUT_MESSAGE,
- WAIT_EVENT_MQ_RECEIVE,
- WAIT_EVENT_MQ_SEND,
- WAIT_EVENT_PARALLEL_BITMAP_SCAN,
- WAIT_EVENT_PARALLEL_CREATE_INDEX_SCAN,
- WAIT_EVENT_PARALLEL_FINISH,
- WAIT_EVENT_PROCARRAY_GROUP_UPDATE,
- WAIT_EVENT_PROMOTE,
- WAIT_EVENT_REPLICATION_ORIGIN_DROP,
- WAIT_EVENT_REPLICATION_SLOT_DROP,
- WAIT_EVENT_SAFE_SNAPSHOT,
- WAIT_EVENT_SYNC_REP
-} WaitEventIPC;
-
-/* ----------
- * Wait Events - Timeout
- *
- * Use this category when a process is waiting for a timeout to expire.
- * ----------
- */
-typedef enum
-{
- WAIT_EVENT_BASE_BACKUP_THROTTLE = PG_WAIT_TIMEOUT,
- WAIT_EVENT_PG_SLEEP,
- WAIT_EVENT_RECOVERY_APPLY_DELAY
-} WaitEventTimeout;
-
-/* ----------
- * Wait Events - IO
- *
- * Use this category when a process is waiting for a IO.
- * ----------
- */
-typedef enum
-{
- WAIT_EVENT_BUFFILE_READ = PG_WAIT_IO,
- WAIT_EVENT_BUFFILE_WRITE,
- WAIT_EVENT_CONTROL_FILE_READ,
- WAIT_EVENT_CONTROL_FILE_SYNC,
- WAIT_EVENT_CONTROL_FILE_SYNC_UPDATE,
- WAIT_EVENT_CONTROL_FILE_WRITE,
- WAIT_EVENT_CONTROL_FILE_WRITE_UPDATE,
- WAIT_EVENT_COPY_FILE_READ,
- WAIT_EVENT_COPY_FILE_WRITE,
- WAIT_EVENT_DATA_FILE_EXTEND,
- WAIT_EVENT_DATA_FILE_FLUSH,
- WAIT_EVENT_DATA_FILE_IMMEDIATE_SYNC,
- WAIT_EVENT_DATA_FILE_PREFETCH,
- WAIT_EVENT_DATA_FILE_READ,
- WAIT_EVENT_DATA_FILE_SYNC,
- WAIT_EVENT_DATA_FILE_TRUNCATE,
- WAIT_EVENT_DATA_FILE_WRITE,
- WAIT_EVENT_DSM_FILL_ZERO_WRITE,
- WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ,
- WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC,
- WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE,
- WAIT_EVENT_LOCK_FILE_CREATE_READ,
- WAIT_EVENT_LOCK_FILE_CREATE_SYNC,
- WAIT_EVENT_LOCK_FILE_CREATE_WRITE,
- WAIT_EVENT_LOCK_FILE_RECHECKDATADIR_READ,
- WAIT_EVENT_LOGICAL_REWRITE_CHECKPOINT_SYNC,
- WAIT_EVENT_LOGICAL_REWRITE_MAPPING_SYNC,
- WAIT_EVENT_LOGICAL_REWRITE_MAPPING_WRITE,
- WAIT_EVENT_LOGICAL_REWRITE_SYNC,
- WAIT_EVENT_LOGICAL_REWRITE_TRUNCATE,
- WAIT_EVENT_LOGICAL_REWRITE_WRITE,
- WAIT_EVENT_RELATION_MAP_READ,
- WAIT_EVENT_RELATION_MAP_SYNC,
- WAIT_EVENT_RELATION_MAP_WRITE,
- WAIT_EVENT_REORDER_BUFFER_READ,
- WAIT_EVENT_REORDER_BUFFER_WRITE,
- WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ,
- WAIT_EVENT_REPLICATION_SLOT_READ,
- WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
- WAIT_EVENT_REPLICATION_SLOT_SYNC,
- WAIT_EVENT_REPLICATION_SLOT_WRITE,
- WAIT_EVENT_SLRU_FLUSH_SYNC,
- WAIT_EVENT_SLRU_READ,
- WAIT_EVENT_SLRU_SYNC,
- WAIT_EVENT_SLRU_WRITE,
- WAIT_EVENT_SNAPBUILD_READ,
- WAIT_EVENT_SNAPBUILD_SYNC,
- WAIT_EVENT_SNAPBUILD_WRITE,
- WAIT_EVENT_TIMELINE_HISTORY_FILE_SYNC,
- WAIT_EVENT_TIMELINE_HISTORY_FILE_WRITE,
- WAIT_EVENT_TIMELINE_HISTORY_READ,
- WAIT_EVENT_TIMELINE_HISTORY_SYNC,
- WAIT_EVENT_TIMELINE_HISTORY_WRITE,
- WAIT_EVENT_TWOPHASE_FILE_READ,
- WAIT_EVENT_TWOPHASE_FILE_SYNC,
- WAIT_EVENT_TWOPHASE_FILE_WRITE,
- WAIT_EVENT_WALSENDER_TIMELINE_HISTORY_READ,
- WAIT_EVENT_WAL_BOOTSTRAP_SYNC,
- WAIT_EVENT_WAL_BOOTSTRAP_WRITE,
- WAIT_EVENT_WAL_COPY_READ,
- WAIT_EVENT_WAL_COPY_SYNC,
- WAIT_EVENT_WAL_COPY_WRITE,
- WAIT_EVENT_WAL_INIT_SYNC,
- WAIT_EVENT_WAL_INIT_WRITE,
- WAIT_EVENT_WAL_READ,
- WAIT_EVENT_WAL_SYNC,
- WAIT_EVENT_WAL_SYNC_METHOD_ASSIGN,
- WAIT_EVENT_WAL_WRITE
-} WaitEventIO;
-
-/* ----------
- * Command type for progress reporting purposes
- * ----------
- */
-typedef enum ProgressCommandType
-{
- PROGRESS_COMMAND_INVALID,
- PROGRESS_COMMAND_VACUUM
-} ProgressCommandType;
-
-#define PGSTAT_NUM_PROGRESS_PARAM 10
-
-/* ----------
- * Shared-memory data structures
- * ----------
- */
-
-
-/*
- * PgBackendSSLStatus
- *
- * For each backend, we keep the SSL status in a separate struct, that
- * is only filled in if SSL is enabled.
- *
- * All char arrays must be null-terminated.
- */
-typedef struct PgBackendSSLStatus
-{
- /* Information about SSL connection */
- int ssl_bits;
- bool ssl_compression;
- char ssl_version[NAMEDATALEN];
- char ssl_cipher[NAMEDATALEN];
- char ssl_client_dn[NAMEDATALEN];
-
- /*
- * serial number is max "20 octets" per RFC 5280, so this size should be
- * fine
- */
- char ssl_client_serial[NAMEDATALEN];
-
- char ssl_issuer_dn[NAMEDATALEN];
-} PgBackendSSLStatus;
-
-
-/* ----------
- * PgBackendStatus
- *
- * Each live backend maintains a PgBackendStatus struct in shared memory
- * showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
- * has no involvement in, or even access to, these structs.
- *
- * Each auxiliary process also maintains a PgBackendStatus struct in shared
- * memory.
- * ----------
- */
-typedef struct PgBackendStatus
-{
- /*
- * To avoid locking overhead, we use the following protocol: a backend
- * increments st_changecount before modifying its entry, and again after
- * finishing a modification. A would-be reader should note the value of
- * st_changecount, copy the entry into private memory, then check
- * st_changecount again. If the value hasn't changed, and if it's even,
- * the copy is valid; otherwise start over. This makes updates cheap
- * while reads are potentially expensive, but that's the tradeoff we want.
- *
- * The above protocol needs the memory barriers to ensure that the
- * apparent order of execution is as it desires. Otherwise, for example,
- * the CPU might rearrange the code so that st_changecount is incremented
- * twice before the modification on a machine with weak memory ordering.
- * This surprising result can lead to bugs.
- */
- int st_changecount;
-
- /* The entry is valid iff st_procpid > 0, unused if st_procpid == 0 */
- int st_procpid;
-
- /* Type of backends */
- BackendType st_backendType;
-
- /* Times when current backend, transaction, and activity started */
- TimestampTz st_proc_start_timestamp;
- TimestampTz st_xact_start_timestamp;
- TimestampTz st_activity_start_timestamp;
- TimestampTz st_state_start_timestamp;
-
- /* Database OID, owning user's OID, connection client address */
- Oid st_databaseid;
- Oid st_userid;
- SockAddr st_clientaddr;
- char *st_clienthostname; /* MUST be null-terminated */
-
- /* Information about SSL connection */
- bool st_ssl;
- PgBackendSSLStatus *st_sslstatus;
-
- /* current state */
- BackendState st_state;
-
- /* application name; MUST be null-terminated */
- char *st_appname;
-
- /*
- * Current command string; MUST be null-terminated. Note that this string
- * possibly is truncated in the middle of a multi-byte character. As
- * activity strings are stored more frequently than read, that allows to
- * move the cost of correct truncation to the display side. Use
- * pgstat_clip_activity() to truncate correctly.
- */
- char *st_activity_raw;
-
- /*
- * Command progress reporting. Any command which wishes can advertise
- * that it is running by setting st_progress_command,
- * st_progress_command_target, and st_progress_param[].
- * st_progress_command_target should be the OID of the relation which the
- * command targets (we assume there's just one, as this is meant for
- * utility commands), but the meaning of each element in the
- * st_progress_param array is command-specific.
- */
- ProgressCommandType st_progress_command;
- Oid st_progress_command_target;
- int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
-} PgBackendStatus;
-
-/*
- * Macros to load and store st_changecount with the memory barriers.
- *
- * pgstat_increment_changecount_before() and
- * pgstat_increment_changecount_after() need to be called before and after
- * PgBackendStatus entries are modified, respectively. This makes sure that
- * st_changecount is incremented around the modification.
- *
- * Also pgstat_save_changecount_before() and pgstat_save_changecount_after()
- * need to be called before and after PgBackendStatus entries are copied into
- * private memory, respectively.
- */
-#define pgstat_increment_changecount_before(beentry) \
- do { \
- beentry->st_changecount++; \
- pg_write_barrier(); \
- } while (0)
-
-#define pgstat_increment_changecount_after(beentry) \
- do { \
- pg_write_barrier(); \
- beentry->st_changecount++; \
- Assert((beentry->st_changecount & 1) == 0); \
- } while (0)
-
-#define pgstat_save_changecount_before(beentry, save_changecount) \
- do { \
- save_changecount = beentry->st_changecount; \
- pg_read_barrier(); \
- } while (0)
-
-#define pgstat_save_changecount_after(beentry, save_changecount) \
- do { \
- pg_read_barrier(); \
- save_changecount = beentry->st_changecount; \
- } while (0)
-
-/* ----------
- * LocalPgBackendStatus
- *
- * When we build the backend status array, we use LocalPgBackendStatus to be
- * able to add new values to the struct when needed without adding new fields
- * to the shared memory. It contains the backend status as a first member.
- * ----------
- */
-typedef struct LocalPgBackendStatus
-{
- /*
- * Local version of the backend status entry.
- */
- PgBackendStatus backendStatus;
-
- /*
- * The xid of the current transaction if available, InvalidTransactionId
- * if not.
- */
- TransactionId backend_xid;
-
- /*
- * The xmin of the current session if available, InvalidTransactionId if
- * not.
- */
- TransactionId backend_xmin;
-} LocalPgBackendStatus;
-
/*
* Working state needed to accumulate per-function-call timing statistics.
*/
@@ -1141,18 +362,18 @@ typedef struct PgStat_FunctionCallUsage
* GUC parameters
* ----------
*/
-extern bool pgstat_track_activities;
extern bool pgstat_track_counts;
extern int pgstat_track_functions;
-extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1164,34 +385,20 @@ extern PgStat_Counter pgStatBlockWriteTime;
* Functions called from postmaster
* ----------
*/
-extern Size BackendStatusShmemSize(void);
-extern void CreateSharedBackendStatus(void);
-
-extern void pgstat_init(void);
-extern int pgstat_start(void);
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_update_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
-extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
-extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
-
+extern void pgstat_reset_shared_counters(const char *target);
+extern void pgstat_reset_single_counter(Oid objectid,
+ PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples);
@@ -1202,87 +409,20 @@ extern void pgstat_report_analyze(Relation rel,
extern void pgstat_report_recovery_conflict(int reason);
extern void pgstat_report_deadlock(void);
+extern void pgstat_clear_snapshot(void);
+
extern void pgstat_initialize(void);
+extern void pgstat_bearray_initialize(void);
extern void pgstat_bestart(void);
-extern void pgstat_report_activity(BackendState state, const char *cmd_str);
-extern void pgstat_report_tempfile(size_t filesize);
-extern void pgstat_report_appname(const char *appname);
-extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
-extern const char *pgstat_get_wait_event(uint32 wait_event_info);
-extern const char *pgstat_get_wait_event_type(uint32 wait_event_info);
-extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
-extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
- int buflen);
-extern const char *pgstat_get_backend_desc(BackendType backendType);
-
-extern void pgstat_progress_start_command(ProgressCommandType cmdtype,
- Oid relid);
-extern void pgstat_progress_update_param(int index, int64 val);
-extern void pgstat_progress_update_multi_param(int nparam, const int *index,
- const int64 *val);
-extern void pgstat_progress_end_command(void);
-
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
extern void pgstat_initstats(Relation rel);
extern char *pgstat_clip_activity(const char *raw_activity);
-
-/* ----------
- * pgstat_report_wait_start() -
- *
- * Called from places where server process needs to wait. This is called
- * to report wait event information. The wait information is stored
- * as 4-bytes where first byte represents the wait event class (type of
- * wait, for different types of wait, refer WaitClass) and the next
- * 3-bytes represent the actual wait event. Currently 2-bytes are used
- * for wait event which is sufficient for current usage, 1-byte is
- * reserved for future usage.
- *
- * NB: this *must* be able to survive being called before MyProc has been
- * initialized.
- * ----------
- */
-static inline void
-pgstat_report_wait_start(uint32 wait_event_info)
-{
- volatile PGPROC *proc = MyProc;
-
- if (!pgstat_track_activities || !proc)
- return;
-
- /*
- * Since this is a four-byte field which is always read and written as
- * four-bytes, updates are atomic.
- */
- proc->wait_event_info = wait_event_info;
-}
-
-/* ----------
- * pgstat_report_wait_end() -
- *
- * Called to report end of a wait.
- *
- * NB: this *must* be able to survive being called before MyProc has been
- * initialized.
- * ----------
- */
-static inline void
-pgstat_report_wait_end(void)
-{
- volatile PGPROC *proc = MyProc;
-
- if (!pgstat_track_activities || !proc)
- return;
-
- /*
- * Since this is a four-byte field which is always read and written as
- * four-bytes, updates are atomic.
- */
- proc->wait_event_info = 0;
-}
+extern PgStat_StatDBEntry *backend_get_db_entry(Oid dbid, bool oneshot);
+extern PgStat_StatTabEntry *backend_get_tab_entry(PgStat_StatDBEntry *dbent, Oid relid, bool oneshot);
/* nontransactional event counts are simple enough to inline */
@@ -1348,21 +488,30 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_update_archiver(const char *xlog, bool failed);
+extern void pgstat_update_bgwriter(void);
+
+extern void pgstat_report_tempfile(size_t filesize);
/* ----------
* Support functions for the SQL-callable functions to
* generate the pgstat* views.
* ----------
*/
-extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
+extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid relid, bool oneshot);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
-extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
-extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
-extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
+
+/* For shared memory allocation/initialize */
+extern Size BackendStatusShmemSize(void);
+extern void CreateSharedBackendStatus(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
#endif /* PGSTAT_H */
diff --git a/src/include/storage/dsm.h b/src/include/storage/dsm.h
index 7c44f4a6e7..c37ec33e9b 100644
--- a/src/include/storage/dsm.h
+++ b/src/include/storage/dsm.h
@@ -26,6 +26,7 @@ typedef struct dsm_segment dsm_segment;
struct PGShmemHeader; /* avoid including pg_shmem.h */
extern void dsm_cleanup_using_control_segment(dsm_handle old_control_handle);
extern void dsm_postmaster_startup(struct PGShmemHeader *);
+extern void dsm_child_init(void);
extern void dsm_backend_shutdown(void);
extern void dsm_detach_all(void);
@@ -33,6 +34,8 @@ extern void dsm_detach_all(void);
extern void dsm_set_control_handle(dsm_handle h);
#endif
+extern bool dsm_is_available(void);
+
/* Functions that create or remove mappings. */
extern dsm_segment *dsm_create(Size size, int flags);
extern dsm_segment *dsm_attach(dsm_handle h);
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 96c7732006..daa269f816 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -219,6 +219,9 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_SHARED_TUPLESTORE,
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
+ LWTRANCHE_STATS_DSA,
+ LWTRANCHE_STATS_DB,
+ LWTRANCHE_STATS_FUNC_TABLE,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 9244a2a7b7..a9b625211b 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
diff --git a/src/test/modules/worker_spi/worker_spi.c b/src/test/modules/worker_spi/worker_spi.c
index c1878dd694..7391e05f37 100644
--- a/src/test/modules/worker_spi/worker_spi.c
+++ b/src/test/modules/worker_spi/worker_spi.c
@@ -290,7 +290,7 @@ worker_spi_main(Datum main_arg)
SPI_finish();
PopActiveSnapshot();
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
pgstat_report_activity(STATE_IDLE, NULL);
}
--
2.16.3
* [PATCH 4/7] Shared-memory based stats collector
@ 2018-11-12 08:26 Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2018-11-12 08:26 UTC (permalink / raw)
This replaces the means to share server statistics numbers from file
to dynamic shared memory. Every backend directly reads and writres to
the stats tables. Stats collector process is removed. Updates of
shared stats happens with the intervals not shorter than 500ms and not
longer than 1s. If the shared memory data is busy and a backend cannot
obtain lock immediately, usually the differences are stashed into
"pending stats" on local memory and merged with the number at the next
chance.
---
src/backend/access/transam/xlog.c | 4 +-
src/backend/postmaster/autovacuum.c | 59 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 24 +-
src/backend/postmaster/pgarch.c | 4 +-
src/backend/postmaster/pgstat.c | 4201 +++++++++++---------------
src/backend/postmaster/postmaster.c | 85 +-
src/backend/replication/logical/tablesync.c | 9 +-
src/backend/replication/logical/worker.c | 4 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/dsm.c | 24 +-
src/backend/storage/ipc/ipci.c | 6 +
src/backend/storage/lmgr/lwlock.c | 3 +
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/tcop/postgres.c | 27 +-
src/backend/utils/adt/pgstatfuncs.c | 50 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 +
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/include/miscadmin.h | 2 +-
src/include/pgstat.h | 437 +--
src/include/storage/dsm.h | 3 +
src/include/storage/lwlock.h | 3 +
src/include/utils/timeout.h | 1 +
src/test/modules/worker_spi/worker_spi.c | 2 +-
25 files changed, 1932 insertions(+), 3043 deletions(-)
inline patch 1: src/backend/access/transam/xlog.c, src/backend/postmaster/autovacuum.c, src/backend/postmaster/bgwriter.c and 22 more (+1932/-3043 lines) download inline-1.patch
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2ab7d804f0..9e45581d89 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8416,9 +8416,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 4cf67873b1..a69ea230fb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -969,7 +969,7 @@ rebuild_database_list(Oid newdb)
PgStat_StatDBEntry *entry;
/* only consider this database if it has a pgstat entry */
- entry = pgstat_fetch_stat_dbentry(newdb);
+ entry = pgstat_fetch_stat_dbentry(newdb, true);
if (entry != NULL)
{
/* we assume it isn't found because the hash was just created */
@@ -978,6 +978,7 @@ rebuild_database_list(Oid newdb)
/* hash_search already filled in the key */
db->adl_score = score++;
/* next_worker is filled in later */
+ pfree(entry);
}
}
@@ -993,7 +994,7 @@ rebuild_database_list(Oid newdb)
* skip databases with no stat entries -- in particular, this gets rid
* of dropped databases
*/
- entry = pgstat_fetch_stat_dbentry(avdb->adl_datid);
+ entry = pgstat_fetch_stat_dbentry(avdb->adl_datid, true);
if (entry == NULL)
continue;
@@ -1005,6 +1006,7 @@ rebuild_database_list(Oid newdb)
db->adl_score = score++;
/* next_worker is filled in later */
}
+ pfree(entry);
}
/* finally, insert all qualifying databases not previously inserted */
@@ -1017,7 +1019,7 @@ rebuild_database_list(Oid newdb)
PgStat_StatDBEntry *entry;
/* only consider databases with a pgstat entry */
- entry = pgstat_fetch_stat_dbentry(avdb->adw_datid);
+ entry = pgstat_fetch_stat_dbentry(avdb->adw_datid, true);
if (entry == NULL)
continue;
@@ -1029,6 +1031,7 @@ rebuild_database_list(Oid newdb)
db->adl_score = score++;
/* next_worker is filled in later */
}
+ pfree(entry);
}
nelems = score;
@@ -1227,7 +1230,7 @@ do_start_worker(void)
continue; /* ignore not-at-risk DBs */
/* Find pgstat entry if any */
- tmp->adw_entry = pgstat_fetch_stat_dbentry(tmp->adw_datid);
+ tmp->adw_entry = pgstat_fetch_stat_dbentry(tmp->adw_datid, true);
/*
* Skip a database with no pgstat entry; it means it hasn't seen any
@@ -1265,16 +1268,22 @@ do_start_worker(void)
break;
}
}
- if (skipit)
- continue;
+ if (!skipit)
+ {
+ /* Remember the db with oldest autovac time. */
+ if (avdb == NULL ||
+ tmp->adw_entry->last_autovac_time <
+ avdb->adw_entry->last_autovac_time)
+ {
+ if (avdb)
+ pfree(avdb->adw_entry);
+ avdb = tmp;
+ }
+ }
- /*
- * Remember the db with oldest autovac time. (If we are here, both
- * tmp->entry and db->entry must be non-null.)
- */
- if (avdb == NULL ||
- tmp->adw_entry->last_autovac_time < avdb->adw_entry->last_autovac_time)
- avdb = tmp;
+ /* Immediately free it if not used */
+ if(avdb != tmp)
+ pfree(tmp->adw_entry);
}
/* Found a database -- process it */
@@ -1963,7 +1972,7 @@ do_autovacuum(void)
* may be NULL if we couldn't find an entry (only happens if we are
* forcing a vacuum for anti-wrap purposes).
*/
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId, true);
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
@@ -2013,7 +2022,7 @@ do_autovacuum(void)
MemoryContextSwitchTo(AutovacMemCxt);
/* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
+ shared = pgstat_fetch_stat_dbentry(InvalidOid, true);
classRel = heap_open(RelationRelationId, AccessShareLock);
@@ -2099,6 +2108,8 @@ do_autovacuum(void)
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
+ if (tabentry)
+ pfree(tabentry);
/* Relations that need work are added to table_oids */
if (dovacuum || doanalyze)
@@ -2178,10 +2189,11 @@ do_autovacuum(void)
/* Fetch the pgstat entry for this table */
tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
shared, dbentry);
-
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
+ if (tabentry)
+ pfree(tabentry);
/* ignore analyze for toast tables */
if (dovacuum)
@@ -2750,12 +2762,10 @@ get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
if (isshared)
{
if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = backend_get_tab_entry(shared, relid, true);
}
else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = backend_get_tab_entry(dbentry, relid, true);
return tabentry;
}
@@ -2787,8 +2797,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
/* use fresh stats */
autovac_refresh_stats();
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
+ shared = pgstat_fetch_stat_dbentry(InvalidOid, true);
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId, true);
/* fetch the relation's relcache entry */
classTup = SearchSysCacheCopy1(RELOID, ObjectIdGetDatum(relid));
@@ -2819,6 +2829,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
&dovacuum, &doanalyze, &wraparound);
+ if (tabentry)
+ pfree(tabentry);
/* ignore ANALYZE for toast tables */
if (classForm->relkind == RELKIND_TOASTVALUE)
@@ -2909,7 +2921,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
}
heap_freetuple(classTup);
-
+ pfree(shared);
+ pfree(dbentry);
return tab;
}
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index e6b6c549de..3fb6badea8 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -267,9 +267,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Update activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_update_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index fe96c41359..d58193774e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -371,7 +371,7 @@ CheckpointerMain(void)
{
checkpoint_requested = false;
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ BgWriterStats.requested_checkpoints++;
}
if (shutdown_requested)
{
@@ -397,7 +397,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ BgWriterStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -515,13 +515,13 @@ CheckpointerMain(void)
CheckArchiveTimeout();
/*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
+ * Update activity statistics. (The reason why we re-use
+ * bgwriter-related code for this is that the bgwriter and
+ * checkpointer used to be just one process. It's probably not worth
+ * the trouble to split the stats support into two independent
+ * functions.)
*/
- pgstat_send_bgwriter();
+ pgstat_update_bgwriter();
/*
* Sleep until we are signaled or it's time for another checkpoint or
@@ -682,9 +682,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Register interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_update_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1284,8 +1284,8 @@ AbsorbFsyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ BgWriterStats.buf_written_backend += CheckpointerShmem->num_backend_writes;
+ BgWriterStats.buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 4342ebdab4..18bd8296b8 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -468,7 +468,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we successfully
* archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_update_archiver(xlog, false);
break; /* out of inner retry loop */
}
@@ -478,7 +478,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we failed to
* archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_update_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index d1fe052abf..a97fbae7a8 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,10 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Statistics collector facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
- *
- * - Add some automatic call for pgstat vacuuming.
- *
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * Statistics data is stored in dynamic shared memory. Every backends
+ * updates and read it individually.
*
* Copyright (c) 2001-2019, PostgreSQL Global Development Group
*
@@ -19,92 +14,59 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "pgstat.h"
#include "access/heapam.h"
#include "access/htup_details.h"
-#include "access/transam.h"
#include "access/twophase_rmgr.h"
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "pg_trace.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
-#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
-#include "utils/tqual.h"
-
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
-
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_STAT_MIN_INTERVAL 500 /* Minimum time between stats data
+ * updates; in milliseconds. */
+#define PGSTAT_STAT_MAX_INTERVAL 1000 /* Maximum time between stats data
+ * updates; in milliseconds. */
/* ----------
* The initial size hints for the hash tables used in the collector.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
+/*
+ * Operation mode of pgstat_get_db_entry.
+ */
+#define PGSTAT_FETCH_SHARED 0
+#define PGSTAT_FETCH_EXCLUSIVE 1
+#define PGSTAT_FETCH_NOWAIT 2
+
+typedef enum
+{
+ PGSTAT_ENTRY_NOT_FOUND,
+ PGSTAT_ENTRY_FOUND,
+ PGSTAT_ENTRY_LOCK_FAILED
+} pg_stat_table_result_status;
/* ----------
* Total number of backends including auxiliary
@@ -132,27 +94,69 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
+/* Shared stats bootstrap infomation */
+typedef struct StatsShmemStruct {
+ dsa_handle stats_dsa_handle;
+ dshash_table_handle db_stats_handle;
+ dsa_pointer global_stats;
+ dsa_pointer archiver_stats;
+} StatsShmemStruct;
+
+
/*
* BgWriter global statistics counters (unused in other processes).
* Stored directly in a stats message structure so it can be sent
* without needing to copy things around. We assume this inits to zeroes.
*/
-PgStat_MsgBgWriter BgWriterStats;
+PgStat_BgWriter BgWriterStats;
/* ----------
* Local data
* ----------
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
+static dshash_table *db_stats;
+static HTAB *snapshot_db_stats;
+static MemoryContext stats_cxt;
-static struct sockaddr_storage pgStatAddr;
+/*
+ * report withholding facility.
+ *
+ * some report items are withholded if required lock is not acquired
+ * immediately.
+ */
+static bool pgstat_pending_recoveryconflict = false;
+static bool pgstat_pending_deadlock = false;
+static bool pgstat_pending_tempfile = false;
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
+/* dshash parameter for each type of table */
+static const dshash_parameters dsh_dbparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatDBEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS_DB
+};
+static const dshash_parameters dsh_tblparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatTabEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS_FUNC_TABLE
+};
+static const dshash_parameters dsh_funcparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatFuncEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS_FUNC_TABLE
+};
/*
* Structures in which backends store per-table info that's waiting to be
@@ -189,18 +193,14 @@ typedef struct TabStatHashEntry
* Hash table for O(1) t_id -> tsa_entry lookup
*/
static HTAB *pgStatTabHash = NULL;
+static HTAB *pgStatPendingTabHash = NULL;
/*
* Backends store per-function info that's waiting to be sent to the collector
* in this hash table (indexed by function OID).
*/
static HTAB *pgStatFunctions = NULL;
-
-/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
- */
-static bool have_function_stats = false;
+static HTAB *pgStatPendingFunctions = NULL;
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -237,6 +237,12 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
+typedef struct
+{
+ dshash_table *tabhash;
+ PgStat_StatDBEntry *dbentry;
+} pgstat_apply_tabstat_context;
+
/*
* Info about current "snapshot" of stats file
*/
@@ -250,23 +256,15 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Cluster wide statistics.
+ * Contains statistics that are not collected per database or per table.
+ * shared_* are the statistics maintained by pgstats and snapshot_* are the
+ * snapshots taken on backends.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
-
-/* Signal handler flags */
-static volatile bool need_exit = false;
-static volatile bool got_SIGHUP = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats *snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats *snapshot_globalStats;
/*
* Total time charged to functions so far in the current backend.
@@ -280,32 +278,41 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-static void pgstat_exit(SIGNAL_ARGS);
+/* functions used in backends */
static void pgstat_beshutdown_hook(int code, Datum arg);
-static void pgstat_sighup_handler(SIGNAL_ARGS);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, int op,
+ pg_stat_table_result_status *status);
+static PgStat_StatTabEntry *pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create);
+static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_db_statsfile(Oid databaseid, dshash_table *tabhash, dshash_table *funchash);
+
+/* functions used in backends */
+static bool backend_snapshot_global_stats(void);
+static PgStat_StatFuncEntry *backend_get_func_etnry(PgStat_StatDBEntry *dbent, Oid funcid, bool oneshot);
static void pgstat_read_current_status(void);
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
+static void pgstat_postmaster_shutdown(int code, Datum arg);
+static void pgstat_apply_pending_tabstats(bool shared, bool force,
+ pgstat_apply_tabstat_context *cxt);
+static bool pgstat_apply_tabstat(pgstat_apply_tabstat_context *cxt,
+ PgStat_TableStatus *entry, bool nowait);
+static void pgstat_merge_tabentry(PgStat_TableStatus *deststat,
+ PgStat_TableStatus *srcstat,
+ bool init);
+static void pgstat_update_funcstats(bool force, PgStat_StatDBEntry *dbentry);
+static void pgstat_reset_all_counters(void);
+static void pgstat_cleanup_recovery_conflict(PgStat_StatDBEntry *dbentry);
+static void pgstat_cleanup_deadlock(PgStat_StatDBEntry *dbentry);
+static void pgstat_cleanup_tempfile(PgStat_StatDBEntry *dbentry);
+
+static inline void pgstat_merge_backendstats_to_funcentry(
+ PgStat_StatFuncEntry *dest, PgStat_BackendFunctionEntry *src, bool init);
+static inline void pgstat_merge_funcentry(
+ PgStat_StatFuncEntry *dest, PgStat_StatFuncEntry *src, bool init);
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
+static void reset_dbentry_counters(PgStat_StatDBEntry *dbentry);
static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
static void pgstat_setup_memcxt(void);
@@ -316,320 +323,16 @@ static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
+static bool pgstat_update_tabentry(dshash_table *tabhash,
+ PgStat_TableStatus *stat, bool nowait);
+static void pgstat_update_dbentry(PgStat_StatDBEntry *dbentry,
+ PgStat_TableStatus *stat);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
- */
-void
-pgstat_init(void)
-{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
-
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
- {
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
- }
-
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
-
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
-
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
-
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
-
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
-
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- test_byte++; /* just make sure variable is changed */
-
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /* If we get here, we have a working socket */
- break;
- }
-
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
-
- /*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
- */
- if (!pg_set_noblock(pgStatSock))
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
- }
-
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
- }
-
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
-}
-
/*
* subroutine for pgstat_reset_all
*/
@@ -678,119 +381,54 @@ pgstat_reset_remove_files(const char *directory)
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats files and on-memory counters. This is currently used only
+ * if WAL recovery is needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
+ pgstat_reset_all_counters();
}
-#ifdef EXEC_BACKEND
-
-/*
- * pgstat_forkexec() -
+/* ----------
+ * pgstat_create_shared_stats() -
*
- * Format up the arglist for, then fork and exec, statistics collector process
+ * create shared stats memory
+ * ----------
*/
-static pid_t
-pgstat_forkexec(void)
+static void
+pgstat_create_shared_stats(void)
{
- char *av[10];
- int ac = 0;
+ MemoryContext oldcontext;
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* lives for the lifetime of the process */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ area = dsa_create(LWTRANCHE_STATS_DSA);
+ dsa_pin_mapping(area);
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
+ db_stats = dshash_create(area, &dsh_dbparams, 0);
+ /* create shared area and write bootstrap information */
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->db_stats_handle =
+ dshash_get_hash_table_handle(db_stats);
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
-
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
-
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
+ /* connect to the memory */
+ snapshot_db_stats = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
+ MemoryContextSwitchTo(oldcontext);
}
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
-}
/* ------------------------------------------------------------
* Public functions used by backends follow
@@ -802,41 +440,107 @@ allow_immediate_pgstat_restart(void)
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
- * transaction stop time as an approximation of current time.
- * ----------
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * This requires taking some locks on the shared statistics hashes and some
+ * of updates may be withholded on lock failure. Pending updates are
+ * retried in later call of this function and finally cleaned up by calling
+ * this function with force = true or PGSTAT_STAT_MAX_INTERVAL milliseconds
+ * was elapsed since last cleanup. On the other hand updates by regular
+ * backends happen with the interval not shorter than
+ * PGSTAT_STAT_MIN_INTERVAL when force = false.
+ *
+ * Returns time in milliseconds until the next update time.
+ *
+ * Note that this is called only when not within a transaction, so it is fair
+ * to use transaction stop time as an approximation of current time.
+ * ----------
*/
-void
-pgstat_report_stat(bool force)
+long
+pgstat_update_stat(bool force)
{
/* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
static TimestampTz last_report = 0;
-
+ static TimestampTz oldest_pending = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
TabStatusArray *tsa;
- int i;
+ pgstat_apply_tabstat_context cxt;
+ bool other_pending_stats = false;
+ long elapsed;
+ long secs;
+ int usecs;
+
+ if (pgstat_pending_recoveryconflict ||
+ pgstat_pending_deadlock ||
+ pgstat_pending_tempfile ||
+ pgStatPendingFunctions)
+ other_pending_stats = true;
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (!other_pending_stats && !pgStatPendingTabHash &&
+ (pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
+ pgStatXactCommit == 0 && pgStatXactRollback == 0)
+ return 0;
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
+
+ if (!force)
+ {
+ /*
+ * Don't update shared stats unless it's been at least
+ * PGSTAT_STAT_MIN_INTERVAL msec since we last updated one.
+ * Returns time to wait in the case.
+ */
+ TimestampDifference(last_report, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs /1000;
+
+ if(elapsed < PGSTAT_STAT_MIN_INTERVAL)
+ {
+ /* we know we have some statistics */
+ if (oldest_pending == 0)
+ oldest_pending = now;
+
+ return PGSTAT_STAT_MIN_INTERVAL - elapsed;
+ }
+
+
+ /*
+ * Don't keep pending stats for longer than PGSTAT_STAT_MAX_INTERVAL.
+ */
+ if (oldest_pending > 0)
+ {
+ TimestampDifference(oldest_pending, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs /1000;
+
+ if(elapsed > PGSTAT_STAT_MAX_INTERVAL)
+ force = true;
+ }
+ }
+
last_report = now;
+ /* setup stats update context*/
+ cxt.dbentry = NULL;
+ cxt.tabhash = NULL;
+
+ /* Forecibly update other stats if any. */
+ if (other_pending_stats)
+ {
+ cxt.dbentry =
+ pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ /* clean up pending statistics if any */
+ if (pgStatPendingFunctions)
+ pgstat_update_funcstats(true, cxt.dbentry);
+ if (pgstat_pending_recoveryconflict)
+ pgstat_cleanup_recovery_conflict(cxt.dbentry);
+ if (pgstat_pending_deadlock)
+ pgstat_cleanup_deadlock(cxt.dbentry);
+ if (pgstat_pending_tempfile)
+ pgstat_cleanup_tempfile(cxt.dbentry);
+ }
+
/*
* Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
* entries it points to. (Should we fail partway through the loop below,
@@ -849,23 +553,55 @@ pgstat_report_stat(bool force)
pgStatTabHash = NULL;
/*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * XX: We cannot lock two dshash entries at once. Since we must keep lock
+ * while tables stats are being updated we have no choice other than
+ * separating jobs for shared table stats and that of egular tables.
+ * Looping over the array twice isapparently ineffcient and more efficient
+ * way is expected.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
+
+ /* The first call of the followings uses dbentry obtained above if any.*/
+ pgstat_apply_pending_tabstats(false, force, &cxt);
+ pgstat_apply_pending_tabstats(true, force, &cxt);
+
+ /* zero out TableStatus structs after use */
+ for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ {
+ MemSet(tsa->tsa_entries, 0,
+ tsa->tsa_used * sizeof(PgStat_TableStatus));
+ tsa->tsa_used = 0;
+ }
+
+ /* record oldest pending update time */
+ if (pgStatPendingTabHash == NULL)
+ oldest_pending = 0;
+ else if (oldest_pending == 0)
+ oldest_pending = now;
+
+ return 0;
+}
+
+/*
+ * Subroutine for pgstat_update_stat.
+ *
+ * Appies table stats in table status array merging with pending stats if any.
+ * If force is true waits until required locks to be acquired. Elsewise stats
+ * merged stats as pending sats and it will be processed in the next chance.
+ */
+static void
+pgstat_apply_pending_tabstats(bool shared, bool force,
+ pgstat_apply_tabstat_context *cxt)
+{
+ static const PgStat_TableCounts all_zeroes;
+ TabStatusArray *tsa;
+ int i;
for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
{
for (i = 0; i < tsa->tsa_used; i++)
{
PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
+ PgStat_TableStatus *pentry = NULL;
/* Shouldn't have any pending transaction-dependent counts */
Assert(entry->trans == NULL);
@@ -878,178 +614,440 @@ pgstat_report_stat(bool force)
sizeof(PgStat_TableCounts)) == 0)
continue;
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Skip if this entry is not match the request */
+ if (entry->t_shared != shared)
+ continue;
+
+ /* if pending update exists, it should be applied along with */
+ if (pgStatPendingTabHash != NULL)
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ pentry = hash_search(pgStatPendingTabHash,
+ (void *) entry, HASH_FIND, NULL);
+
+ if (pentry)
+ {
+ /* merge new update into pending updates */
+ pgstat_merge_tabentry(pentry, entry, false);
+ entry = pentry;
+ }
+ }
+
+ /* try to apply the merged stats */
+ if (pgstat_apply_tabstat(cxt, entry, !force))
+ {
+ /* succeeded. remove it if it was pending stats */
+ if (pentry && entry != pentry)
+ hash_search(pgStatPendingTabHash,
+ (void *) pentry, HASH_REMOVE, NULL);
+ }
+ else if (!pentry)
+ {
+ /* failed and there was no pending entry, create new one. */
+ bool found;
+
+ if (pgStatPendingTabHash == NULL)
+ {
+ HASHCTL ctl;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(PgStat_TableStatus);
+ pgStatPendingTabHash =
+ hash_create("pgstat pending table stats hash",
+ TABSTAT_QUANTUM,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ pentry = hash_search(pgStatPendingTabHash,
+ (void *) entry, HASH_ENTER, &found);
+ Assert (!found);
+
+ *pentry = *entry;
}
}
- /* zero out TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ }
+
+ /* if any pending stats exists, try to clean it up */
+ if (pgStatPendingTabHash != NULL)
+ {
+ HASH_SEQ_STATUS pstat;
+ PgStat_TableStatus *pentry;
+
+ hash_seq_init(&pstat, pgStatPendingTabHash);
+ while((pentry = (PgStat_TableStatus *) hash_seq_search(&pstat)) != NULL)
+ {
+ /* Skip if this entry is not match the request */
+ if (pentry->t_shared != shared)
+ continue;
+
+ /* apply pending entry and remove on success */
+ if (pgstat_apply_tabstat(cxt, pentry, !force))
+ hash_search(pgStatPendingTabHash,
+ (void *) pentry, HASH_REMOVE, NULL);
+ }
+
+ /* destroy the hash if no entry is left */
+ if (hash_get_num_entries(pgStatPendingTabHash) == 0)
+ {
+ hash_destroy(pgStatPendingTabHash);
+ pgStatPendingTabHash = NULL;
+ }
+ }
+
+ if (cxt->tabhash)
+ dshash_detach(cxt->tabhash);
+ if (cxt->dbentry)
+ dshash_release_lock(db_stats, cxt->dbentry);
+ cxt->tabhash = NULL;
+ cxt->dbentry = NULL;
+}
+
+
+/*
+ * pgstat_apply_tabstat: update shared stats entry using given entry
+ *
+ * If nowait is true, just returns false on lock failure. Dshashes for table
+ * and function stats are kept attached and stored in ctx. The caller must
+ * detach them after use.
+ */
+bool
+pgstat_apply_tabstat(pgstat_apply_tabstat_context *cxt,
+ PgStat_TableStatus *entry, bool nowait)
+{
+ Oid dboid = entry->t_shared ? InvalidOid : MyDatabaseId;
+ int table_mode = PGSTAT_FETCH_EXCLUSIVE;
+ bool updated = false;
+
+ if (nowait)
+ table_mode |= PGSTAT_FETCH_NOWAIT;
+
+ /*
+ * We need to keep lock on dbentries for regular tables to avoid race
+ * condition with drop database. So we hold it in the context variable. We
+ * don't need that for shared tables.
+ */
+ if (!cxt->dbentry)
+ cxt->dbentry = pgstat_get_db_entry(dboid, table_mode, NULL);
+
+ /* we cannot acquire lock, just return */
+ if (!cxt->dbentry)
+ return false;
+
+ /* attach shared stats table if not yet */
+ if (!cxt->tabhash)
+ {
+ /* apply database stats */
+ if (!entry->t_shared)
+ {
+ /* Update database-wide stats */
+ cxt->dbentry->n_xact_commit += pgStatXactCommit;
+ cxt->dbentry->n_xact_rollback += pgStatXactRollback;
+ cxt->dbentry->n_block_read_time += pgStatBlockReadTime;
+ cxt->dbentry->n_block_write_time += pgStatBlockWriteTime;
+ pgStatXactCommit = 0;
+ pgStatXactRollback = 0;
+ pgStatBlockReadTime = 0;
+ pgStatBlockWriteTime = 0;
+ }
+
+ cxt->tabhash =
+ dshash_attach(area, &dsh_tblparams, cxt->dbentry->tables, 0);
}
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * If we have access to the required data, try update table stats first.
+ * Update database stats only if the first step suceeded.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
+ if (pgstat_update_tabentry(cxt->tabhash, entry, nowait))
+ {
+ pgstat_update_dbentry(cxt->dbentry, entry);
+ updated = true;
+ }
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ return updated;
}
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * pgstat_merge_tabentry: subroutine for pgstat_update_stat
+ *
+ * Merge srcstat into deststat. Existing value in deststat is cleard if
+ * init is true.
*/
static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+pgstat_merge_tabentry(PgStat_TableStatus *deststat,
+ PgStat_TableStatus *srcstat,
+ bool init)
{
- int n;
- int len;
+ Assert (deststat != srcstat);
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
- */
- if (OidIsValid(tsmsg->m_databaseid))
- {
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
- pgStatXactCommit = 0;
- pgStatXactRollback = 0;
- pgStatBlockReadTime = 0;
- pgStatBlockWriteTime = 0;
- }
+ if (init)
+ deststat->t_counts = srcstat->t_counts;
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ PgStat_TableCounts *dest = &deststat->t_counts;
+ PgStat_TableCounts *src = &srcstat->t_counts;
+
+ dest->t_numscans += src->t_numscans;
+ dest->t_tuples_returned += src->t_tuples_returned;
+ dest->t_tuples_fetched += src->t_tuples_fetched;
+ dest->t_tuples_inserted += src->t_tuples_inserted;
+ dest->t_tuples_updated += src->t_tuples_updated;
+ dest->t_tuples_deleted += src->t_tuples_deleted;
+ dest->t_tuples_hot_updated += src->t_tuples_hot_updated;
+ dest->t_truncated |= src->t_truncated;
+
+ /* If table was truncated, first reset the live/dead counters */
+ if (src->t_truncated)
+ {
+ dest->t_delta_live_tuples = 0;
+ dest->t_delta_dead_tuples = 0;
+ }
+ dest->t_delta_live_tuples += src->t_delta_live_tuples;
+ dest->t_delta_dead_tuples += src->t_delta_dead_tuples;
+ dest->t_changed_tuples += src->t_changed_tuples;
+ dest->t_blocks_fetched += src->t_blocks_fetched;
+ dest->t_blocks_hit += src->t_blocks_hit;
}
-
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
-
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
}
-
+
/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+ * pgstat_update_funcstats: subroutine for pgstat_update_stat
+ *
+ * updates a function stat
*/
static void
-pgstat_send_funcstats(void)
+pgstat_update_funcstats(bool force, PgStat_StatDBEntry *dbentry)
{
/* we assume this inits to all zeroes: */
static const PgStat_FunctionCounts all_zeroes;
+ pg_stat_table_result_status status = 0;
+ dshash_table *funchash;
+ bool nowait = !force;
+ bool release_db = false;
+ int table_op = PGSTAT_FETCH_EXCLUSIVE;
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
+ if (pgStatFunctions == NULL && pgStatPendingFunctions == NULL)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
+ if (nowait)
+ table_op += PGSTAT_FETCH_NOWAIT;
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ /* find the shared function stats table */
+ if (!dbentry)
{
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ dbentry = pgstat_get_db_entry(MyDatabaseId, table_op, &status);
+ release_db = true;
}
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
+ /* lock failure, return. */
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
- have_function_stats = false;
+ /* create hash if not yet */
+ if (dbentry->functions == DSM_HANDLE_INVALID)
+ {
+ funchash = dshash_create(area, &dsh_funcparams, 0);
+ dbentry->functions = dshash_get_hash_table_handle(funchash);
+ }
+ else
+ funchash = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+
+ /*
+ * First, we empty the transaction stats. Just move numbers to pending
+ * stats if any. Elsewise try to directly update the shared stats but
+ * create a new pending entry on lock failure.
+ */
+ if (pgStatFunctions)
+ {
+ HASH_SEQ_STATUS fstat;
+ PgStat_BackendFunctionEntry *bestat;
+
+ hash_seq_init(&fstat, pgStatFunctions);
+ while ((bestat = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ {
+ bool found;
+ bool init = false;
+ PgStat_StatFuncEntry *funcent = NULL;
+
+ /* Skip it if no counts accumulated since last time */
+ if (memcmp(&bestat->f_counts, &all_zeroes,
+ sizeof(PgStat_FunctionCounts)) == 0)
+ continue;
+
+ /* find pending entry */
+ if (pgStatPendingFunctions)
+ funcent = (PgStat_StatFuncEntry *)
+ hash_search(pgStatPendingFunctions,
+ (void *) &(bestat->f_id), HASH_FIND, NULL);
+
+ if (!funcent)
+ {
+ /* pending entry not found, find shared stats entry */
+ funcent = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert_extended(funchash,
+ (void *) &(bestat->f_id),
+ &found, nowait);
+ if (funcent)
+ init = !found;
+ else
+ {
+ /* no shared stats entry. create a new pending one */
+ funcent = (PgStat_StatFuncEntry *)
+ hash_search(pgStatPendingFunctions,
+ (void *) &(bestat->f_id), HASH_ENTER, NULL);
+ init = true;
+ }
+ }
+ Assert (funcent != NULL);
+
+ pgstat_merge_backendstats_to_funcentry(funcent, bestat, init);
+
+ /* reset used counts */
+ MemSet(&bestat->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ }
+ }
+
+ /* Second, apply pending stats numbers to shared table */
+ if (pgStatPendingFunctions)
+ {
+ HASH_SEQ_STATUS fstat;
+ PgStat_StatFuncEntry *pendent;
+
+ hash_seq_init(&fstat, pgStatPendingFunctions);
+ while ((pendent = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ {
+ PgStat_StatFuncEntry *funcent;
+ bool found;
+
+ funcent = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert_extended(funchash,
+ (void *) &(pendent->functionid),
+ &found, nowait);
+ if (funcent)
+ {
+ pgstat_merge_funcentry(pendent, funcent, !found);
+ hash_search(pgStatPendingFunctions,
+ (void *) &(pendent->functionid), HASH_REMOVE, NULL);
+ }
+ }
+
+ /* destroy the hsah if no entry remains */
+ if (hash_get_num_entries(pgStatPendingFunctions) == 0)
+ {
+ hash_destroy(pgStatPendingFunctions);
+ pgStatPendingFunctions = NULL;
+ }
+ }
+
+ if (release_db)
+ dshash_release_lock(db_stats, dbentry);
}
+/*
+ * pgstat_merge_backendstats_to_funcentry: subroutine for
+ * pgstat_update_funcstats
+ *
+ * Merges BackendFunctionEntry into StatFuncEntry
+ */
+static inline void
+pgstat_merge_backendstats_to_funcentry(PgStat_StatFuncEntry *dest,
+ PgStat_BackendFunctionEntry *src,
+ bool init)
+{
+ if (init)
+ {
+ /*
+ * If it's a new function entry, initialize counters to the values
+ * we just got.
+ */
+ dest->f_numcalls = src->f_counts.f_numcalls;
+ dest->f_total_time =
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_total_time);
+ dest->f_self_time =
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_self_time);
+ }
+ else
+ {
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ dest->f_numcalls += src->f_counts.f_numcalls;
+ dest->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_total_time);
+ dest->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(src->f_counts.f_self_time);
+ }
+}
+
+/*
+ * pgstat_merge_funcentry: subroutine for pgstat_update_funcstats
+ *
+ * Merges two StatFuncEntrys
+ */
+static inline void
+pgstat_merge_funcentry(PgStat_StatFuncEntry *dest, PgStat_StatFuncEntry *src,
+ bool init)
+{
+ if (init)
+ {
+ /*
+ * If it's a new function entry, initialize counters to the values
+ * we just got.
+ */
+ dest->f_numcalls = src->f_numcalls;
+ dest->f_total_time = src->f_total_time;
+ dest->f_self_time = src->f_self_time;
+ }
+ else
+ {
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ dest->f_numcalls += src->f_numcalls;
+ dest->f_total_time += src->f_total_time;
+ dest->f_self_time += src->f_self_time;
+ }
+}
+
+
/* ----------
* pgstat_vacuum_stat() -
*
- * Will tell the collector about objects he can get rid of.
+ * Remove objects he can get rid of.
* ----------
*/
void
pgstat_vacuum_stat(void)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
+ HTAB *oidtab;
+ dshash_table *dshtable;
+ dshash_seq_status dshstat;
PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
- int len;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* we don't collect statistics under standalone mode */
+ if (!IsUnderPostmaster)
return;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* If not done for this transaction, take a snapshot of stats */
+ if (!backend_snapshot_global_stats())
+ return;
/*
* Read pg_database and make a list of OIDs of all existing databases
*/
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ oidtab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
/*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
+ * Search the database hash table for dead databases and drop them
+ * from the hash.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+
+ dshash_seq_init(&dshstat, db_stats, false, true);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&dshstat)) != NULL)
{
Oid dbid = dbentry->databaseid;
@@ -1057,148 +1055,86 @@ pgstat_vacuum_stat(void)
/* the DB entry for shared tables (with InvalidOid) is never dropped */
if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
+ hash_search(oidtab, (void *) &dbid, HASH_FIND, NULL) == NULL)
pgstat_drop_database(dbid);
}
/* Clean up */
- hash_destroy(htab);
+ hash_destroy(oidtab);
/*
* Lookup our own database entry; if not found, nothing more to do.
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ if (!dbentry)
return;
-
+
/*
* Similarly to above, make a list of all known relations in this DB.
*/
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
+ oidtab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
/*
* Check for all tables listed in stats hashtable if they still exist.
+ * Stats cache is useless here so directly search the shared hash.
*/
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ dshtable = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&dshstat, dshtable, false, true);
+ while ((tabentry = (PgStat_StatTabEntry *) dshash_seq_next(&dshstat)) != NULL)
{
Oid tabid = tabentry->tableid;
CHECK_FOR_INTERRUPTS();
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ if (hash_search(oidtab, (void *) &tabid, HASH_FIND, NULL) != NULL)
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ /* Not there, so purge this table */
+ dshash_delete_entry(dshtable, tabentry);
}
+ dshash_detach(dshtable);
/* Clean up */
- hash_destroy(htab);
+ hash_destroy(oidtab);
/*
* Now repeat the above steps for functions. However, we needn't bother
* in the common case where no function stats are being collected.
*/
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+ dshtable = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ oidtab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_init(&dshstat, dshtable, false, true);
+ while ((funcentry = (PgStat_StatFuncEntry *) dshash_seq_next(&dshstat)) != NULL)
{
Oid funcid = funcentry->functionid;
CHECK_FOR_INTERRUPTS();
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
+ if (hash_search(oidtab, (void *) &funcid, HASH_FIND, NULL) != NULL)
continue;
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
+ /* Not there, so remove this function */
+ dshash_delete_entry(dshtable, funcentry);
}
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
+ hash_destroy(oidtab);
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
+ dshash_detach(dshtable);
}
+ dshash_release_lock(db_stats, dbentry);
}
-/* ----------
+/*
* pgstat_collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
- * into a temporary hash table. Caller should hash_destroy the result
- * when done with it. (However, we make the table in CurrentMemoryContext
- * so that it will be freed properly in event of an error.)
- * ----------
+ * into a temporary hash table. Caller should hash_destroy the result after
+ * use. (However, we make the table in CurrentMemoryContext so that it will
+ * be freed properly in event of an error.)
*/
static HTAB *
pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
@@ -1245,62 +1181,54 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
+ * Remove entry for the database that we just dropped.
+ *
+ * If some stats update happens after this, this entry will re-created but
+ * we will still clean the dead DB eventually via future invocations of
+ * pgstat_vacuum_stat().
* ----------
*/
+
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ Assert (OidIsValid(databaseid));
+ Assert(db_stats);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Lookup the database in the hashtable with exclusive lock.
+ */
+ dbentry = pgstat_get_db_entry(databaseid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ /*
+ * If found, remove it (along with the db statfile).
+ */
+ if (dbentry)
+ {
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(tbl);
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(tbl);
+ }
+
+ dshash_delete_entry(db_stats, (void *)dbentry);
+ }
}
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
-
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-}
-#endif /* NOT_USED */
-
-
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1309,20 +1237,51 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStat_StatDBEntry *dbentry;
+ pg_stat_table_result_status status;
- if (pgStatSock == PGINVALID_SOCKET)
+ Assert(db_stats);
+
+ /*
+ * Lookup the database in the hashtable. Nothing to do if not there.
+ */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, &status);
+
+ if (!dbentry)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * We simply throw away all the database's table entries by recreating a
+ * new hash table for them.
+ */
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(t);
+ dbentry->tables = DSM_HANDLE_INVALID;
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(t);
+ dbentry->functions = DSM_HANDLE_INVALID;
+ }
+
+ /*
+ * Reset database-level stats, too. This creates empty hash tables for
+ * tables and functions.
+ */
+ reset_dbentry_counters(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1331,29 +1290,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ Assert(db_stats);
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ memset(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = GetCurrentTimestamp();
+ }
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ /* Reset the global background writer statistics for the cluster. */
+ memset(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
+
+ LWLockRelease(StatsLock);
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1362,17 +1329,90 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStat_StatDBEntry *dbentry;
+
- if (pgStatSock == PGINVALID_SOCKET)
+ Assert(db_stats);
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ if (!dbentry)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore it if not */
+ if (type == RESET_TABLE)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_delete_key(t, (void *) &objoid);
+ }
+
+ if (type == RESET_FUNCTION && dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_delete_key(t, (void *) &objoid);
+ }
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * pgstat_reset_all_counters: subroutine for pgstat_reset_all
+ *
+ * clear all counters on shared memory
+ */
+static void
+pgstat_reset_all_counters(void)
+{
+ dshash_seq_status dshstat;
+ PgStat_StatDBEntry *dbentry;
+
+ Assert (db_stats);
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ dshash_seq_init(&dshstat, db_stats, false, true);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&dshstat)) != NULL)
+ {
+ /*
+ * We simply throw away all the database's table hashes
+ */
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(t);
+ dbentry->tables = DSM_HANDLE_INVALID;
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *t =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(t);
+ dbentry->functions = DSM_HANDLE_INVALID;
+ }
+
+ /*
+ * Reset database-level stats, too. This creates empty hash tables
+ * for tables and functions.
+ */
+ reset_dbentry_counters(dbentry);
+ dshash_release_lock(db_stats, dbentry);
+
+ }
+
+ /*
+ * Reset global counters
+ */
+ memset(shared_globalStats, 0, sizeof(*shared_globalStats));
+ memset(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_globalStats->stat_reset_timestamp =
+ shared_archiverStats->stat_reset_timestamp = GetCurrentTimestamp();
+
+ LWLockRelease(StatsLock);
}
/* ----------
@@ -1386,48 +1426,75 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ /*
+ * Store the last autovacuum time in the database's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
- pgstat_send(&msg, sizeof(msg));
+ dbentry->last_autovac_time = GetCurrentTimestamp();
+
+ dshash_release_lock(db_stats, dbentry);
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Repot about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ table = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ tabentry = pgstat_get_tab_entry(table, tableoid, true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->vacuum_count++;
+ }
+ dshash_release_lock(table, tabentry);
+ dshash_detach(table);
+ dshash_release_lock(db_stats, dbentry);
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1438,9 +1505,14 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
return;
/*
@@ -1469,114 +1541,228 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ table = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ tabentry = pgstat_get_tab_entry(table, RelationGetRelid(rel), true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+ dshash_release_lock(table, tabentry);
+ dshash_detach(table);
+ dshash_release_lock(db_stats, dbentry);
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
+static int pending_conflict_tablespace = 0;
+static int pending_conflict_lock = 0;
+static int pending_conflict_snapshot = 0;
+static int pending_conflict_bufferpin = 0;
+static int pending_conflict_startup_deadlock = 0;
+
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbentry;
+ pg_stat_table_result_status status;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ pgstat_pending_recoveryconflict = true;
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ pending_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ pending_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ pending_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ pending_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ pending_conflict_startup_deadlock++;
+ break;
+ }
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ pgstat_cleanup_recovery_conflict(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * clean up function for pending recovery conflicts
+ */
+static void
+pgstat_cleanup_recovery_conflict(PgStat_StatDBEntry *dbentry)
+{
+ dbentry->n_conflict_tablespace += pending_conflict_tablespace;
+ dbentry->n_conflict_lock += pending_conflict_lock;
+ dbentry->n_conflict_snapshot += pending_conflict_snapshot;
+ dbentry->n_conflict_bufferpin += pending_conflict_bufferpin;
+ dbentry->n_conflict_startup_deadlock += pending_conflict_startup_deadlock;
+
+ pending_conflict_tablespace = 0;
+ pending_conflict_lock = 0;
+ pending_conflict_snapshot = 0;
+ pending_conflict_bufferpin = 0;
+ pending_conflict_startup_deadlock = 0;
+
+ pgstat_pending_recoveryconflict = false;
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
+static int pending_deadlocks = 0;
+
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbentry;
+ pg_stat_table_result_status status;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ pending_deadlocks++;
+ pgstat_pending_deadlock = true;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ pgstat_cleanup_deadlock(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
+}
+
+/*
+ * clean up function for pending dead locks
+ */
+static void
+pgstat_cleanup_deadlock(PgStat_StatDBEntry *dbentry)
+{
+ dbentry->n_deadlocks += pending_deadlocks;
+ pending_deadlocks = 0;
+ pgstat_pending_deadlock = false;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
+static size_t pending_filesize = 0;
+static size_t pending_files = 0;
+
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbentry;
+ pg_stat_table_result_status status;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
-}
+ if (filesize > 0) /* Is't there a case where filesize is really 0? */
+ {
+ pgstat_pending_tempfile = true;
+ pending_filesize += filesize; /* needs check overflow */
+ pending_files++;
+ }
-
-/* ----------
- * pgstat_ping() -
- *
- * Send some junk data to the collector to increase traffic.
- * ----------
- */
-void
-pgstat_ping(void)
-{
- PgStat_MsgDummy msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!pgstat_pending_tempfile)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ pgstat_cleanup_tempfile(dbentry);
+
+ dshash_release_lock(db_stats, dbentry);
}
-/* ----------
- * pgstat_send_inquiry() -
- *
- * Notify collector that we need fresh data.
- * ----------
+/*
+ * clean up function for temporary files
*/
static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
+pgstat_cleanup_tempfile(PgStat_StatDBEntry *dbentry)
{
- PgStat_MsgInquiry msg;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
+ dbentry->n_temp_bytes += pending_filesize;
+ dbentry->n_temp_files += pending_files;
+ pending_filesize = 0;
+ pending_files = 0;
+ pgstat_pending_tempfile = false;
+
}
-
/*
* Initialize function call usage data.
* Called by the executor before invoking a function.
@@ -1692,9 +1878,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1716,6 +1899,15 @@ pgstat_initstats(Relation rel)
Oid rel_id = rel->rd_id;
char relkind = rel->rd_rel->relkind;
+ Assert(db_stats);
+
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ {
+ /* We're not counting at all */
+ rel->pgstat_info = NULL;
+ return;
+ }
+
/* We only count stats for things that have storage */
if (!(relkind == RELKIND_RELATION ||
relkind == RELKIND_MATVIEW ||
@@ -1727,13 +1919,6 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- {
- /* We're not counting at all */
- rel->pgstat_info = NULL;
- return;
- }
-
/*
* If we already set up this relation in the current transaction, nothing
* to do.
@@ -2377,34 +2562,6 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
rec->tuples_inserted + rec->tuples_updated;
}
-
-/* ----------
- * pgstat_fetch_stat_dbentry() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
- * ----------
- */
-PgStat_StatDBEntry *
-pgstat_fetch_stat_dbentry(Oid dbid)
-{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
-}
-
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
@@ -2417,47 +2574,28 @@ pgstat_fetch_stat_dbentry(Oid dbid)
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* Lookup our database, then look in its table hash table. */
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId, false);
+ if (dbentry == NULL)
+ return NULL;
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = backend_get_tab_entry(dbentry, relid, false);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ dbentry = pgstat_fetch_stat_dbentry(InvalidOid, false);
+ if (dbentry == NULL)
+ return NULL;
+
+ tabentry = backend_get_tab_entry(dbentry, relid, false);
+ if (tabentry != NULL)
+ return tabentry;
return NULL;
}
@@ -2476,18 +2614,14 @@ pgstat_fetch_stat_funcentry(Oid func_id)
PgStat_StatDBEntry *dbentry;
PgStat_StatFuncEntry *funcentry = NULL;
- /* load the stats file if needed */
- backend_read_statsfile();
+ /* Lookup our database, then find the requested function */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_SHARED, NULL);
+ if (dbentry == NULL)
+ return NULL;
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
+ funcentry = backend_get_func_etnry(dbentry, func_id, false);
+ dshash_release_lock(db_stats, dbentry);
return funcentry;
}
@@ -2562,9 +2696,11 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ if (!backend_snapshot_global_stats())
+ return NULL;
- return &archiverStats;
+ return snapshot_archiverStats;
}
@@ -2579,9 +2715,11 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ if (!backend_snapshot_global_stats())
+ return NULL;
- return &globalStats;
+ return snapshot_globalStats;
}
@@ -2771,7 +2909,7 @@ pgstat_initialize(void)
}
/* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -2963,7 +3101,7 @@ pgstat_beshutdown_hook(int code, Datum arg)
* during failed backend starts might never get counted.)
*/
if (OidIsValid(MyDatabaseId))
- pgstat_report_stat(true);
+ pgstat_update_stat(true);
/*
* Clear my status entry, following the protocol of bumping st_changecount
@@ -3230,7 +3368,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -4150,96 +4289,68 @@ pgstat_get_backend_desc(BackendType backendType)
* ------------------------------------------------------------
*/
-
/* ----------
- * pgstat_setheader() -
+ * pgstat_update_archiver() -
*
- * Set common header fields in a statistics message
+ * Update the stats data about the WAL file that we successfully archived or
+ * failed to archive.
* ----------
*/
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
+void
+pgstat_update_archiver(const char *xlog, bool failed)
{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
+ if (failed)
{
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
+ /* Failed archival attempt */
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = GetCurrentTimestamp();
+ }
+ else
+ {
+ /* Successful archival operation */
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = GetCurrentTimestamp();
+ }
}
/* ----------
- * pgstat_send_archiver() -
+ * pgstat_update_bgwriter() -
*
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Update bgwriter statistics
* ----------
*/
void
-pgstat_send_archiver(const char *xlog, bool failed)
-{
- PgStat_MsgArchiver msg;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* ----------
- * pgstat_send_bgwriter() -
- *
- * Send bgwriter statistics to the collector
- * ----------
- */
-void
-pgstat_send_bgwriter(void)
+pgstat_update_bgwriter(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *s = &BgWriterStats;
/*
* This function can be called even if nothing at all has happened. In
* this case, avoid sending a completely empty message to the stats
* collector.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += s->timed_checkpoints;
+ shared_globalStats->requested_checkpoints += s->requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += s->checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += s->checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += s->buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += s->buf_written_clean;
+ shared_globalStats->maxwritten_clean += s->maxwritten_clean;
+ shared_globalStats->buf_written_backend += s->buf_written_backend;
+ shared_globalStats->buf_fsync_backend += s->buf_fsync_backend;
+ shared_globalStats->buf_alloc += s->buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4247,299 +4358,15 @@ pgstat_send_bgwriter(void)
MemSet(&BgWriterStats, 0, sizeof(BgWriterStats));
}
-
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, pgstat_sighup_handler);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, pgstat_exit);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- /*
- * Identify myself via ps
- */
- init_ps_display("stats collector", "", "", "");
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do recognize got_SIGHUP inside
- * the inner loop, which means that such interrupts will get serviced but
- * the latch won't get cleared until next time there is a break in the
- * action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (need_exit)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * need_exit becomes set.
- */
- while (!need_exit)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (got_SIGHUP)
- {
- got_SIGHUP = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry((PgStat_MsgInquiry *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat((PgStat_MsgTabstat *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge((PgStat_MsgTabpurge *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb((PgStat_MsgDropdb *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter((PgStat_MsgResetcounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(
- (PgStat_MsgResetsharedcounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(
- (PgStat_MsgResetsinglecounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac((PgStat_MsgAutovacStart *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum((PgStat_MsgVacuum *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze((PgStat_MsgAnalyze *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver((PgStat_MsgArchiver *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter((PgStat_MsgBgWriter *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat((PgStat_MsgFuncstat *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge((PgStat_MsgFuncpurge *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict((PgStat_MsgRecoveryConflict *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock((PgStat_MsgDeadlock *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile((PgStat_MsgTempFile *) &msg, len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- exit(0);
-}
-
-
-/* SIGQUIT signal handler for collector process */
-static void
-pgstat_exit(SIGNAL_ARGS)
-{
- int save_errno = errno;
-
- need_exit = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
-}
-
-/* SIGHUP handler for collector process */
-static void
-pgstat_sighup_handler(SIGNAL_ARGS)
-{
- int save_errno = errno;
-
- got_SIGHUP = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
-}
-
/*
- * Subroutine to clear stats in a database entry
+ * Subroutine to reset stats in a shared database entry
*
* Tables and functions hashes are initialized to empty.
*/
static void
reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
{
- HASHCTL hash_ctl;
+ dshash_table *tbl;
dbentry->n_xact_commit = 0;
dbentry->n_xact_rollback = 0;
@@ -4565,20 +4392,17 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
dbentry->stat_reset_timestamp = GetCurrentTimestamp();
dbentry->stats_timestamp = 0;
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
+ Assert(dbentry->tables == DSM_HANDLE_INVALID);
+ tbl = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables = dshash_get_hash_table_handle(tbl);
+ dshash_detach(tbl);
+
+ Assert(dbentry->functions == DSM_HANDLE_INVALID);
+ /* we create function hash as needed */
+
+ dbentry->snapshot_tables = NULL;
+ dbentry->snapshot_functions = NULL;
}
/*
@@ -4587,47 +4411,76 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
* Else, return NULL.
*/
static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
+pgstat_get_db_entry(Oid databaseid, int op, pg_stat_table_result_status *status)
{
PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
+ bool nowait = ((op & PGSTAT_FETCH_NOWAIT) != 0);
+ bool lock_acquired = true;
+ bool found = true;
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
+ if (!IsUnderPostmaster)
return NULL;
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
+ /* Lookup or create the hash table entry for this database */
+ if (op & PGSTAT_FETCH_EXCLUSIVE)
+ {
+ result = (PgStat_StatDBEntry *)
+ dshash_find_or_insert_extended(db_stats, &databaseid,
+ &found, nowait);
+ if (result == NULL)
+ lock_acquired = false;
+ else if (!found)
+ {
+ /*
+ * If not found, initialize the new one. This creates empty hash
+ * tables for tables and functions, too.
+ */
+ reset_dbentry_counters(result);
+ }
+ }
+ else
+ {
+ result = (PgStat_StatDBEntry *)
+ dshash_find_extended(db_stats, &databaseid, true, nowait,
+ &lock_acquired);
+ if (result == NULL)
+ found = false;
+ }
+
+ /* Set return status if requested */
+ if (status)
+ {
+ if (!lock_acquired)
+ {
+ Assert(nowait);
+ *status = PGSTAT_ENTRY_LOCK_FAILED;
+ }
+ else if (!found)
+ *status = PGSTAT_ENTRY_NOT_FOUND;
+ else
+ *status = PGSTAT_ENTRY_FOUND;
+ }
return result;
}
-
/*
* Lookup the hash table entry for the specified table. If no hash
* table entry exists, initialize it, if the create parameter is true.
* Else, return NULL.
*/
static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
+pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create)
{
PgStat_StatTabEntry *result;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
/* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
+ if (create)
+ result = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(table, &tableoid, &found);
+ else
+ result = (PgStat_StatTabEntry *) dshash_find(table, &tableoid, false);
if (!create && !found)
return NULL;
@@ -4663,29 +4516,23 @@ pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
+ dshash_seq_status hstat;
PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ /* should be called from postmaster */
+ Assert(!IsUnderPostmaster);
+
elog(DEBUG2, "writing stats file \"%s\"", statfile);
/*
@@ -4704,7 +4551,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4716,32 +4563,29 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_init(&hstat, db_stats, false, false);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&hstat)) != NULL)
{
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_db_statsfile(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
@@ -4784,16 +4628,6 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
}
/*
@@ -4801,15 +4635,14 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
* of length len.
*/
static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
+get_dbstat_filename(bool tempname, Oid databaseid,
char *filename, int len)
{
int printed;
/* NB -- pgstat_reset_remove_files knows about the pattern this uses */
printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
databaseid,
tempname ? "tmp" : "stat");
if (printed >= len)
@@ -4827,10 +4660,10 @@ get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
+ dshash_seq_status tstat;
+ dshash_seq_status fstat;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
FILE *fpout;
@@ -4839,9 +4672,10 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
int rc;
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
+ dshash_table *tbl;
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -4868,23 +4702,30 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&tstat, tbl, false, false);
+ while ((tabentry = (PgStat_StatTabEntry *) dshash_seq_next(&tstat)) != NULL)
{
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ dshash_detach(tbl);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ tbl = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_seq_init(&fstat, tbl, false, false);
+ while ((funcentry = (PgStat_StatFuncEntry *) dshash_seq_next(&fstat)) != NULL)
+ {
+ fputc('F', fpout);
+ rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ dshash_detach(tbl);
}
/*
@@ -4919,47 +4760,30 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
}
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * Reads in some existing statistics collector files into the shared stats
+ * hash.
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+ dshash_table *tblstats = NULL;
+ dshash_table *funcstats = NULL;
+
+ /* should be called from postmaster */
+ Assert(!IsUnderPostmaster);
/*
* The tables will live in pgStatLocalContext.
@@ -4967,28 +4791,18 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
pgstat_setup_memcxt();
/*
- * Create the DB hashtable
+ * Create the DB hashtable and global stas area
*/
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ /* Hold lock so that no other process looks empty stats */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ pgstat_create_shared_stats();
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp = shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
@@ -5002,11 +4816,12 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ LWLockRelease(StatsLock);
+ return;
}
/*
@@ -5015,7 +4830,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5023,11 +4838,12 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ memset(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
@@ -5038,17 +4854,17 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
* file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
* an unusual scenario.
*/
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
+ shared_globalStats->stats_timestamp = 0;
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ memset(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
@@ -5068,7 +4884,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
fpin) != offsetof(PgStat_StatDBEntry, tables))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5077,21 +4893,23 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+ dbentry = (PgStat_StatDBEntry *)
+ dshash_find_or_insert(db_stats, (void *) &dbbuf.databaseid,
+ &found);
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(db_stats, dbentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
+ dbentry->tables = DSM_HANDLE_INVALID;
+ dbentry->functions = DSM_HANDLE_INVALID;
+ dbentry->snapshot_tables = NULL;
+ dbentry->snapshot_functions = NULL;
/*
* In the collector, disregard the timestamp we read from the
@@ -5099,54 +4917,26 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
* stats file immediately upon the first request from any
* backend.
*/
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ dbentry->stats_timestamp = 0;
/*
* If requested, read the data from the database-specific
* file. Otherwise we just leave the hashtables empty.
*/
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
-
+ tblstats = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables = dshash_get_hash_table_handle(tblstats);
+ /* we don't create function hash at the present */
+ dshash_release_lock(db_stats, dbentry);
+ pgstat_read_db_statsfile(dbentry->databaseid,
+ tblstats, funcstats);
+ dshash_detach(tblstats);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5154,36 +4944,62 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
}
done:
+ LWLockRelease(StatsLock);
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+void
+StatsShmemInit(void)
+{
+ bool found;
+
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+
+ /* Load saved data if any */
+ pgstat_read_statsfiles();
+
+ /* need to be called before dsm shutodwn */
+ before_shmem_exit(pgstat_postmaster_shutdown, (Datum) 0);
+ }
+}
+
+static void
+pgstat_postmaster_shutdown(int code, Datum arg)
+{
+ /* we trash the stats on crash */
+ if (code == 0)
+ pgstat_write_statsfiles();
+}
+
/* ----------
* pgstat_read_db_statsfile() -
*
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the permanent statistics collector file and create shared
+ * statistics tables. The file is removed afer reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_db_statsfile(Oid databaseid,
+ dshash_table *tabhash, dshash_table *funchash)
{
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
@@ -5194,7 +5010,10 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ /* should be called from postmaster */
+ Assert(!IsUnderPostmaster);
+
+ get_dbstat_filename(false, databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
@@ -5208,7 +5027,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5221,7 +5040,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5241,7 +5060,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5253,19 +5072,21 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (tabhash == NULL)
break;
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(tabhash,
+ (void *) &tabbuf.tableid, &found);
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(tabhash, tabentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
memcpy(tabentry, &tabbuf, sizeof(tabbuf));
+ dshash_release_lock(tabhash, tabentry);
break;
/*
@@ -5275,7 +5096,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5287,19 +5108,20 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (funchash == NULL)
break;
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ funcentry = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert(funchash,
+ (void *) &funcbuf.functionid, &found);
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
memcpy(funcentry, &funcbuf, sizeof(funcbuf));
+ dshash_release_lock(funchash, funcentry);
break;
/*
@@ -5309,7 +5131,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5319,276 +5141,290 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
done:
FreeFile(fpin);
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
/* ----------
- * pgstat_read_db_statsfile_timestamp() -
+ * backend_clean_snapshot_callback() -
*
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
+ * This is usually called with arg = NULL when the memory context where the
+ * current snapshot has been taken. Don't bother releasing memory in the
+ * case.
* ----------
*/
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
+static void
+backend_clean_snapshot_callback(void *arg)
{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
+ if (arg != NULL)
{
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
+ /* explicitly called, so explicitly free resources */
+ if (snapshot_globalStats)
+ pfree(snapshot_globalStats);
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
+ if (snapshot_archiverStats)
+ pfree(snapshot_archiverStats);
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
+ if (snapshot_db_stats)
{
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
+ HASH_SEQ_STATUS seq;
+ PgStat_StatDBEntry *dbent;
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
+ hash_seq_init(&seq, snapshot_db_stats);
+ while ((dbent = hash_seq_search(&seq)) != NULL)
+ {
+ if (dbent->snapshot_tables)
+ hash_destroy(dbent->snapshot_tables);
+ if (dbent->snapshot_functions)
+ hash_destroy(dbent->snapshot_functions);
+ }
+ hash_destroy(snapshot_db_stats);
}
}
-done:
- FreeFile(fpin);
- return true;
+ /* mark as the resource are not allocated */
+ snapshot_globalStats = NULL;
+ snapshot_archiverStats = NULL;
+ snapshot_db_stats = NULL;
}
/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
+ * create_local_stats_hash() -
+ *
+ * Creates a dynahash used for table/function stats cache.
*/
-static void
-backend_read_statsfile(void)
+static HTAB *
+create_local_stats_hash(const char *name, size_t keysize, size_t entrysize,
+ int nentries)
{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
+ HTAB *result;
+ HASHCTL ctl;
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
+ /* Create the hash in the stats context */
+ ctl.keysize = keysize;
+ ctl.entrysize = entrysize;
+ ctl.hcxt = stats_cxt;
+ result = hash_create(name, nentries, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ return result;
}
+/*
+ * snapshot_statentry() - Find an entriy from source dshash.
+ *
+ * Returns the entry for key or NULL if not found. If dest is not null, uses
+ * *dest as local cache, which is created in the same shape with the given
+ * dshash when *dest is NULL. In both cases the result is cached in the hash
+ * and the same entry is returned to subsequent calls for the same key.
+ *
+ * Otherwise returned entry is a copy that is palloc'ed in the current memory
+ * context. Its content may differ for every request.
+ *
+ * If dshash is NULL, temporaralily attaches dsh_handle instead.
+ */
+static void *
+snapshot_statentry(HTAB **dest, const char *hashname,
+ dshash_table *dshash, dshash_table_handle dsh_handle,
+ const dshash_parameters *dsh_params, Oid key)
+{
+ void *lentry = NULL;
+ size_t keysize = dsh_params->key_size;
+ size_t entrysize = dsh_params->entry_size;
+
+ if (dest)
+ {
+ /* caches the result entry */
+ bool found;
+
+ /*
+ * Create new hash with arbitrary initial entries since we don't know
+ * how this hash will grow.
+ */
+ if (!*dest)
+ {
+ Assert(hashname);
+ *dest = create_local_stats_hash(hashname, keysize, entrysize, 32);
+ }
+
+ lentry = hash_search(*dest, &key, HASH_ENTER, &found);
+ if (!found)
+ {
+ dshash_table *t = dshash;
+ void *sentry;
+
+ if (!t)
+ t = dshash_attach(area, dsh_params, dsh_handle, NULL);
+
+ sentry = dshash_find(t, &key, false);
+
+ /*
+ * We expect that the stats for specified database exists in most
+ * cases.
+ */
+ if (!sentry)
+ {
+ hash_search(*dest, &key, HASH_REMOVE, NULL);
+ if (!dshash)
+ dshash_detach(t);
+ return NULL;
+ }
+ memcpy(lentry, sentry, entrysize);
+ dshash_release_lock(t, sentry);
+
+ if (!dshash)
+ dshash_detach(t);
+ }
+ }
+ else
+ {
+ /*
+ * The caller don't want caching. Just make a copy of the entry then
+ * return.
+ */
+ dshash_table *t = dshash;
+ void *sentry;
+
+ if (!t)
+ t = dshash_attach(area, dsh_params, dsh_handle, NULL);
+
+ sentry = dshash_find(t, &key, false);
+ if (sentry)
+ {
+ lentry = palloc(entrysize);
+ memcpy(lentry, sentry, entrysize);
+ dshash_release_lock(t, sentry);
+ }
+
+ if (!dshash)
+ dshash_detach(t);
+ }
+
+ return lentry;
+}
+
+/*
+ * backend_snapshot_global_stats() -
+ *
+ * Makes a local copy of global stats if not already done. They will be kept
+ * until pgstat_clear_snapshot() is called or the end of the current memory
+ * context (typically TopTransactionContext). Returns false if the shared
+ * stats is not created yet.
+ */
+static bool
+backend_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext = CurrentMemoryContext;
+ MemoryContextCallback *mcxt_cb;
+
+ /* Nothing to do if already done */
+ if (snapshot_globalStats)
+ return true;
+
+ Assert(snapshot_archiverStats == NULL);
+
+ /*
+ * The snapshot lives within the current top transaction if any, or the
+ * current memory context liftime otherwise.
+ */
+ if (IsTransactionState())
+ oldcontext = MemoryContextSwitchTo(TopTransactionContext);
+
+ /* Remember for stats memory allocation later */
+ stats_cxt = CurrentMemoryContext;
+
+ /* global stats can be just copied */
+ snapshot_globalStats = palloc(sizeof(PgStat_GlobalStats));
+ memcpy(snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ snapshot_archiverStats = palloc(sizeof(PgStat_ArchiverStats));
+ memcpy(snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+
+ /* set the timestamp of this snapshot */
+ snapshot_globalStats->stats_timestamp = GetCurrentTimestamp();
+
+ /* register callback to clear snapshot */
+ mcxt_cb = (MemoryContextCallback *)palloc(sizeof(MemoryContextCallback));
+ mcxt_cb->func = backend_clean_snapshot_callback;
+ mcxt_cb->arg = NULL;
+ MemoryContextRegisterResetCallback(CurrentMemoryContext, mcxt_cb);
+ MemoryContextSwitchTo(oldcontext);
+
+ return true;
+}
+
+/* ----------
+ * pgstat_fetch_stat_dbentry() -
+ *
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end. If onshot is true, they are not cached and returned
+ * in a palloc'ed memory.
+ */
+PgStat_StatDBEntry *
+pgstat_fetch_stat_dbentry(Oid dbid, bool oneshot)
+{
+ /* take a local snapshot if we don't have one */
+ char *hashname = "local database stats hash";
+ PgStat_StatDBEntry *dbentry;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* If not done for this transaction, take a snapshot of global stats */
+ if (!backend_snapshot_global_stats())
+ return NULL;
+
+ dbentry = snapshot_statentry(oneshot ? NULL : &snapshot_db_stats,
+ hashname, db_stats, 0, &dsh_dbparams,
+ dbid);
+
+ return dbentry;
+}
+
+/* ----------
+ * backend_get_tab_entry() -
+ *
+ * Find table stats entry on backends. The returned entries are cached until
+ * transaction end. If onshot is true, they are not cached and returned in a
+ * palloc'ed memory.
+ */
+PgStat_StatTabEntry *
+backend_get_tab_entry(PgStat_StatDBEntry *dbent, Oid reloid, bool oneshot)
+{
+ /* take a local snapshot if we don't have one */
+ char *hashname = "local table stats hash";
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return snapshot_statentry(oneshot ? NULL : &dbent->snapshot_tables,
+ hashname, NULL, dbent->tables, &dsh_tblparams,
+ reloid);
+}
+
+/* ----------
+ * backend_get_func_entry() -
+ *
+ * Find function stats entry on backends. The returned entries are cached
+ * until transaction end. If onshot is true, they are not cached and returned
+ * in a palloc'ed memory.
+ */
+static PgStat_StatFuncEntry *
+backend_get_func_etnry(PgStat_StatDBEntry *dbent, Oid funcid, bool oneshot)
+{
+ char *hashname = "local table stats hash";
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ if (dbent->functions == DSM_HANDLE_INVALID)
+ return NULL;
+
+ return snapshot_statentry(oneshot ? NULL : &dbent->snapshot_tables,
+ hashname, NULL, dbent->functions, &dsh_funcparams,
+ funcid);
+}
/* ----------
* pgstat_setup_memcxt() -
@@ -5619,6 +5455,8 @@ pgstat_setup_memcxt(void)
void
pgstat_clear_snapshot(void)
{
+ int param = 0; /* only the address is significant */
+
/* Release memory, if any was allocated */
if (pgStatLocalContext)
MemoryContextDelete(pgStatLocalContext);
@@ -5628,717 +5466,112 @@ pgstat_clear_snapshot(void)
pgStatDBHash = NULL;
localBackendStatusTable = NULL;
localNumBackends = 0;
+
+ /*
+ * the parameter inform the function that it is not called from
+ * MemoryContextCallback
+ */
+ backend_clean_snapshot_callback(¶m);
}
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
+static bool
+pgstat_update_tabentry(dshash_table *tabhash, PgStat_TableStatus *stat,
+ bool nowait)
{
- PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ bool found;
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
+ if (tabhash == NULL)
+ return false;
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert_extended(tabhash, (void *) &(stat->t_id),
+ &found, nowait);
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
+ /* failed to acquire lock */
+ if (tabentry == NULL)
+ return false;
+
+ if (!found)
{
/*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
+ * If it's a new table entry, initialize counters to the values we
+ * just got.
*/
+ tabentry->numscans = stat->t_counts.t_numscans;
+ tabentry->tuples_returned = stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched = stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted = stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated = stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted = stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated = stat->t_counts.t_tuples_hot_updated;
+ tabentry->n_live_tuples = stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples = stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze = stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched = stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit = stat->t_counts.t_blocks_hit;
+
+ tabentry->vacuum_timestamp = 0;
+ tabentry->vacuum_count = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_count = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->analyze_count = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ tabentry->autovac_analyze_count = 0;
}
- else if (msg->clock_time < dbentry->stats_timestamp)
+ else
{
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
/*
- * Add per-table stats to the per-database entry, too.
+ * Otherwise add the values to the existing entry.
*/
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetshared() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
+ tabentry->numscans += stat->t_counts.t_numscans;
+ tabentry->tuples_returned += stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched += stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted += stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated += stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted += stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated += stat->t_counts.t_tuples_hot_updated;
+ /* If table was truncated, first reset the live/dead counters */
+ if (stat->t_counts.t_truncated)
{
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
+ tabentry->n_live_tuples = 0;
+ tabentry->n_dead_tuples = 0;
}
+ tabentry->n_live_tuples += stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples += stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze += stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched += stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit += stat->t_counts.t_blocks_hit;
}
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
+
+ dshash_release_lock(tabhash, tabentry);
+
+ return true;
}
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
+pgstat_update_dbentry(PgStat_StatDBEntry *dbentry, PgStat_TableStatus *stat)
{
/*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
+ * Add per-table stats to the per-database entry, too.
*/
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
+ dbentry->n_tuples_returned += stat->t_counts.t_tuples_returned;
+ dbentry->n_tuples_fetched += stat->t_counts.t_tuples_fetched;
+ dbentry->n_tuples_inserted += stat->t_counts.t_tuples_inserted;
+ dbentry->n_tuples_updated += stat->t_counts.t_tuples_updated;
+ dbentry->n_tuples_deleted += stat->t_counts.t_tuples_deleted;
+ dbentry->n_blocks_fetched += stat->t_counts.t_blocks_fetched;
+ dbentry->n_blocks_hit += stat->t_counts.t_blocks_hit;
}
+
/*
* Convert a potentially unsafely truncated activity string (see
* PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 65eab02b3e..dd293a79f0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -255,7 +255,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -502,7 +501,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1298,12 +1296,6 @@ PostmasterMain(int argc, char *argv[])
whereToSendOutput = DestNone;
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1752,11 +1744,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2591,8 +2578,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -2923,8 +2908,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -2991,13 +2974,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3072,22 +3048,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3546,22 +3506,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3757,8 +3701,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3797,8 +3739,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -3999,8 +3940,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -4973,18 +4912,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5097,12 +5024,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5972,7 +5893,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6025,8 +5945,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6258,7 +6176,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index d87cf8afe5..9408f87614 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -128,7 +128,7 @@ finish_sync_worker(void)
if (IsTransactionState())
{
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(true);
}
/* And flush all writes. */
@@ -144,6 +144,9 @@ finish_sync_worker(void)
/* Find the main apply worker and signal it. */
logicalrep_worker_wakeup(MyLogicalRepWorker->subid, InvalidOid);
+ /* clean up retained statistics */
+ pgstat_update_stat(true);
+
/* Stop gracefully */
proc_exit(0);
}
@@ -525,7 +528,7 @@ process_syncing_tables_for_apply(XLogRecPtr current_lsn)
if (started_tx)
{
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
}
}
@@ -863,7 +866,7 @@ LogicalRepSyncTableStart(XLogRecPtr *origin_startpos)
MyLogicalRepWorker->relstate,
MyLogicalRepWorker->relstate_lsn);
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
/*
* We want to do the table data sync in a single transaction.
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index de23ced9af..e4e2ad7b39 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -493,7 +493,7 @@ apply_handle_commit(StringInfo s)
replorigin_session_origin_timestamp = commit_data.committime;
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
store_flush_position(commit_data.end_lsn);
}
@@ -1327,6 +1327,8 @@ LogicalRepApplyLoop(XLogRecPtr last_received)
}
send_feedback(last_received, requestReply, requestReply);
+
+ pgstat_update_stat(false);
}
}
}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 273e2f385f..d8d0ad2487 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1990,7 +1990,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ BgWriterStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2098,7 +2098,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2288,7 +2288,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2296,7 +2296,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c
index cab7ae74ca..c7c248878a 100644
--- a/src/backend/storage/ipc/dsm.c
+++ b/src/backend/storage/ipc/dsm.c
@@ -197,6 +197,15 @@ dsm_postmaster_startup(PGShmemHeader *shim)
dsm_control->maxitems = maxitems;
}
+/*
+ * clear dsm_init state on child start.
+ */
+void
+dsm_child_init(void)
+{
+ dsm_init_done = false;
+}
+
/*
* Determine whether the control segment from the previous postmaster
* invocation still exists. If so, remove the dynamic shared memory
@@ -423,6 +432,15 @@ dsm_set_control_handle(dsm_handle h)
}
#endif
+/*
+ * Return if dsm feature is available on this process.
+ */
+bool
+dsm_is_available(void)
+{
+ return dsm_control != NULL;
+}
+
/*
* Create a new dynamic shared memory segment.
*
@@ -440,8 +458,7 @@ dsm_create(Size size, int flags)
uint32 i;
uint32 nitems;
- /* Unsafe in postmaster (and pointless in a stand-alone backend). */
- Assert(IsUnderPostmaster);
+ Assert(dsm_is_available());
if (!dsm_init_done)
dsm_backend_startup();
@@ -537,8 +554,7 @@ dsm_attach(dsm_handle h)
uint32 i;
uint32 nitems;
- /* Unsafe in postmaster (and pointless in a stand-alone backend). */
- Assert(IsUnderPostmaster);
+ Assert(dsm_is_available());
if (!dsm_init_done)
dsm_backend_startup();
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 2849e47d99..6417559cb0 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -148,6 +148,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -279,8 +280,13 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
/* Initialize dynamic shared memory facilities. */
if (!IsUnderPostmaster)
+ {
dsm_postmaster_startup(shim);
+ /* Stats collector uses dynamic shared memory */
+ StatsShmemInit();
+ }
+
/*
* Now give loadable modules a chance to set up their shmem allocations
*/
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 81dac45ae5..979478e2e5 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -521,6 +521,9 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_TBM, "tbm");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
+ LWLockRegisterTranche(LWTRANCHE_STATS_DSA, "stats table dsa");
+ LWLockRegisterTranche(LWTRANCHE_STATS_DB, "db stats");
+ LWLockRegisterTranche(LWTRANCHE_STATS_FUNC_TABLE, "table/func stats");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index db47843229..97eccb35d3 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -49,3 +49,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
LogicalRepWorkerLock 43
CLogTruncationLock 44
+StatsLock 45
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0c0891b33e..8b9142461a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3159,6 +3159,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_update_stat(true);
+ }
}
@@ -3733,6 +3739,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4173,9 +4180,17 @@ PostgresMain(int argc, char *argv[],
}
else
{
- ProcessCompletedNotifies();
- pgstat_report_stat(false);
+ long stats_timeout;
+ ProcessCompletedNotifies();
+
+ stats_timeout = pgstat_update_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle", false);
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4210,7 +4225,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4218,6 +4233,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 053bb73863..6eac39fb57 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -33,7 +33,7 @@
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
+extern PgStat_BgWriter bgwriterStats;
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
@@ -1176,7 +1176,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_xact_commit);
@@ -1192,7 +1192,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_xact_rollback);
@@ -1208,7 +1208,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_blocks_fetched);
@@ -1224,7 +1224,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_blocks_hit);
@@ -1240,7 +1240,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_returned);
@@ -1256,7 +1256,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_fetched);
@@ -1272,7 +1272,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_inserted);
@@ -1288,7 +1288,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_updated);
@@ -1304,7 +1304,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_tuples_deleted);
@@ -1319,7 +1319,7 @@ pg_stat_get_db_stat_reset_time(PG_FUNCTION_ARGS)
TimestampTz result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = dbentry->stat_reset_timestamp;
@@ -1337,7 +1337,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = dbentry->n_temp_files;
@@ -1353,7 +1353,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = dbentry->n_temp_bytes;
@@ -1368,7 +1368,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_tablespace);
@@ -1383,7 +1383,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_lock);
@@ -1398,7 +1398,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_snapshot);
@@ -1413,7 +1413,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_bufferpin);
@@ -1428,7 +1428,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_conflict_startup_deadlock);
@@ -1443,7 +1443,7 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (
@@ -1463,7 +1463,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
int64 result;
PgStat_StatDBEntry *dbentry;
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = (int64) (dbentry->n_deadlocks);
@@ -1479,7 +1479,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
PgStat_StatDBEntry *dbentry;
/* convert counter from microsec to millisec for display */
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = ((double) dbentry->n_block_read_time) / 1000.0;
@@ -1495,7 +1495,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
PgStat_StatDBEntry *dbentry;
/* convert counter from microsec to millisec for display */
- if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
+ if ((dbentry = pgstat_fetch_stat_dbentry(dbid, false)) == NULL)
result = 0;
else
result = ((double) dbentry->n_block_write_time) / 1000.0;
@@ -1850,6 +1850,9 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
/* Get statistics about the archiver process */
archiver_stats = pgstat_fetch_stat_archiver();
+ if (archiver_stats == NULL)
+ PG_RETURN_NULL();
+
/* Fill values and NULLs */
values[0] = Int64GetDatum(archiver_stats->archived_count);
if (*(archiver_stats->last_archived_wal) == '\0')
@@ -1879,6 +1882,5 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(
- heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index fd51934aaf..994351ac2d 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -32,6 +32,7 @@ volatile sig_atomic_t QueryCancelPending = false;
volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile sig_atomic_t ConfigReloadPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 7415c4faab..626a4326a4 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -629,6 +630,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1240,6 +1243,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3e1c3863c4..25b3b2a079 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 63a7653457..49131a6d5b 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -82,6 +82,7 @@ extern PGDLLIMPORT volatile sig_atomic_t InterruptPending;
extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -403,7 +404,6 @@ typedef enum
CheckpointerProcess,
WalWriterProcess,
WalReceiverProcess,
-
NUM_AUXPROCTYPES /* Must be last! */
} AuxProcType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f299d1d601..1ad77fb20f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -13,6 +13,7 @@
#include "datatype/timestamp.h"
#include "fmgr.h"
+#include "lib/dshash.h"
#include "libpq/pqcomm.h"
#include "port/atomics.h"
#include "portability/instr_time.h"
@@ -41,32 +42,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -115,13 +90,6 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_blocks_hit;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -180,271 +148,23 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
/* ----------
- * PgStat_MsgHdr The common message header
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgHdr
+typedef struct PgStat_BgWriter
{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter buf_alloc;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
@@ -485,79 +205,6 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
* Statistic collector data structures follow
*
@@ -601,10 +248,13 @@ typedef struct PgStat_StatDBEntry
/*
* tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * the handles and pointers out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables;
+ dshash_table_handle functions;
+ /* for snapshot tables */
+ HTAB *snapshot_tables;
+ HTAB *snapshot_functions;
} PgStat_StatDBEntry;
@@ -1136,13 +786,15 @@ extern bool pgstat_track_counts;
extern int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1154,34 +806,20 @@ extern PgStat_Counter pgStatBlockWriteTime;
* Functions called from postmaster
* ----------
*/
-extern Size BackendStatusShmemSize(void);
-extern void CreateSharedBackendStatus(void);
-
-extern void pgstat_init(void);
-extern int pgstat_start(void);
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_update_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
-extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
-extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
-
+extern void pgstat_reset_shared_counters(const char *target);
+extern void pgstat_reset_single_counter(Oid objectid,
+ PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples);
@@ -1192,6 +830,8 @@ extern void pgstat_report_analyze(Relation rel,
extern void pgstat_report_recovery_conflict(int reason);
extern void pgstat_report_deadlock(void);
+extern void pgstat_clear_snapshot(void);
+
extern void pgstat_initialize(void);
extern void pgstat_bestart(void);
@@ -1219,6 +859,9 @@ extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
extern void pgstat_initstats(Relation rel);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern PgStat_StatDBEntry *backend_get_db_entry(Oid dbid, bool oneshot);
+extern HTAB *backend_snapshot_all_db_entries(void);
+extern PgStat_StatTabEntry *backend_get_tab_entry(PgStat_StatDBEntry *dbent, Oid relid, bool oneshot);
/* ----------
* pgstat_report_wait_start() -
@@ -1338,15 +981,15 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_update_archiver(const char *xlog, bool failed);
+extern void pgstat_update_bgwriter(void);
/* ----------
* Support functions for the SQL-callable functions to
* generate the pgstat* views.
* ----------
*/
-extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
+extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid relid, bool oneshot);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
@@ -1355,4 +998,14 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
+
+/* For shared memory allocation/initialize */
+extern Size BackendStatusShmemSize(void);
+extern void CreateSharedBackendStatus(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
#endif /* PGSTAT_H */
diff --git a/src/include/storage/dsm.h b/src/include/storage/dsm.h
index 7c44f4a6e7..c37ec33e9b 100644
--- a/src/include/storage/dsm.h
+++ b/src/include/storage/dsm.h
@@ -26,6 +26,7 @@ typedef struct dsm_segment dsm_segment;
struct PGShmemHeader; /* avoid including pg_shmem.h */
extern void dsm_cleanup_using_control_segment(dsm_handle old_control_handle);
extern void dsm_postmaster_startup(struct PGShmemHeader *);
+extern void dsm_child_init(void);
extern void dsm_backend_shutdown(void);
extern void dsm_detach_all(void);
@@ -33,6 +34,8 @@ extern void dsm_detach_all(void);
extern void dsm_set_control_handle(dsm_handle h);
#endif
+extern bool dsm_is_available(void);
+
/* Functions that create or remove mappings. */
extern dsm_segment *dsm_create(Size size, int flags);
extern dsm_segment *dsm_attach(dsm_handle h);
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 96c7732006..daa269f816 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -219,6 +219,9 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_SHARED_TUPLESTORE,
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
+ LWTRANCHE_STATS_DSA,
+ LWTRANCHE_STATS_DB,
+ LWTRANCHE_STATS_FUNC_TABLE,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 9244a2a7b7..a9b625211b 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
diff --git a/src/test/modules/worker_spi/worker_spi.c b/src/test/modules/worker_spi/worker_spi.c
index c1878dd694..7391e05f37 100644
--- a/src/test/modules/worker_spi/worker_spi.c
+++ b/src/test/modules/worker_spi/worker_spi.c
@@ -290,7 +290,7 @@ worker_spi_main(Datum main_arg)
SPI_finish();
PopActiveSnapshot();
CommitTransactionCommand();
- pgstat_report_stat(false);
+ pgstat_update_stat(false);
pgstat_report_activity(STATE_IDLE, NULL);
}
--
2.16.3
* [PATCH v24 4/5] Shared-memory based stats collector
@ 2019-02-21 03:44 Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2019-02-21 03:44 UTC (permalink / raw)
Previously activity statistics is shared via files on disk. Every
backend sends the numbers to the stats collector process via a socket.
It makes snapshots as a set of files on disk with a certain interval
then every backend reads them as necessary. It worked fine for
comparatively small set of statistics but the set is under the
pressure to growing up and the file size has reached the order of
megabytes. To deal with larger statistics set, this patch let backends
directly share the statistics via shared memory.
---
doc/src/sgml/monitoring.sgml | 6 +-
src/backend/postmaster/autovacuum.c | 12 +-
src/backend/postmaster/pgstat.c | 4652 ++++++++----------
src/backend/postmaster/postmaster.c | 85 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 1 +
src/backend/tcop/postgres.c | 26 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 +
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 441 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/timeout.h | 1 +
14 files changed, 2132 insertions(+), 3112 deletions(-)
inline patch 1: doc/src/sgml/monitoring.sgml, src/backend/postmaster/autovacuum.c, src/backend/postmaster/pgstat.c and 11 more (+2132/-3112 lines) download inline-1.patch
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index a3c5f86b7e..eb94dec119 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -53,7 +53,6 @@ postgres 15554 0.0 0.0 57536 1184 ? Ss 18:02 0:00 postgres: back
postgres 15555 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: checkpointer
postgres 15556 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: walwriter
postgres 15557 0.0 0.0 58504 2244 ? Ss 18:02 0:00 postgres: autovacuum launcher
-postgres 15558 0.0 0.0 17512 1068 ? Ss 18:02 0:00 postgres: stats collector
postgres 15582 0.0 0.0 58772 3080 ? Ss 18:04 0:00 postgres: joe runbug 127.0.0.1 idle
postgres 15606 0.0 0.0 58772 3052 ? Ss 18:07 0:00 postgres: tgl regression [local] SELECT waiting
postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl regression [local] idle in transaction
@@ -65,9 +64,8 @@ postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl
master server process. The command arguments
shown for it are the same ones used when it was launched. The next five
processes are background worker processes automatically launched by the
- master process. (The <quote>stats collector</quote> process will not be present
- if you have set the system not to start the statistics collector; likewise
- the <quote>autovacuum launcher</quote> process can be disabled.)
+ master process. (The <quote>autovacuum launcher</quote> process will not
+ be present if you have set the system not to start it.)
Each of the remaining
processes is a server process handling one client connection. Each such
process sets its command line display in the form
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index c1dd8168ca..e07c7cb311 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -1961,15 +1961,15 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
+ /* Start a transaction so our commands have one to play into. */
+ StartTransactionCommand();
+
/*
* may be NULL if we couldn't find an entry (only happens if we are
* forcing a vacuum for anti-wrap purposes).
*/
dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- /* Start a transaction so our commands have one to play into. */
- StartTransactionCommand();
-
/*
* Clean up any dead statistics collector entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
@@ -2752,12 +2752,10 @@ get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
if (isshared)
{
if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = pgstat_fetch_stat_tabentry_extended(shared, relid);
}
else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = pgstat_fetch_stat_tabentry_extended(dbentry, relid);
return tabentry;
}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 8299d2a435..bcf8c6f371 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,23 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Statistics collector facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects per-table and per-function usage statistics of all backends on
+ * shared memory. pg_count_*() and friends interfaces stores activity of
+ * every backend during a transaction. Then pgstat_flush_stat() is called at
+ * the end of a transaction to flush out the local numbers to shared memory.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, we update shared stats no more
+ * often than intervals of PGSTAT_STAT_MIN_INTERVAL(500ms). Still it is
+ * possible that a backend cannot flush all or a part of local numbers
+ * immediately, we postpone updates and try the next chance after the
+ * interval of PGSTAT_STAT_RETRY_INTERVAL(100ms), but they are not kept
+ * longer than PGSTAT_STAT_MAX_INTERVAL(1000ms).
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses stats collector creates the area then load the
+ * stored stats file if any, and the last process at shutdown writes the
+ * shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2019, PostgreSQL Global Development Group
*
@@ -19,18 +27,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,66 +36,39 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
-#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
-
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
+#define PGSTAT_STAT_MIN_INTERVAL 500 /* Minimum time between stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_STAT_RETRY_INTERVAL 100 /* Retry interval between after
+ * elapsed PGSTAT_MIN_INTERVAL*/
+#define PGSTAT_STAT_MAX_INTERVAL 1000 /* Maximum time between stats data
+ * updates; in milliseconds. */
/* ----------
* The initial size hints for the hash tables used in the collector.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -115,6 +84,19 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+/*
+ * Operation mode and return code of pgstat_get_db_entry.
+ */
+#define PGSTAT_SHARED 0
+#define PGSTAT_EXCLUSIVE 1
+#define PGSTAT_NOWAIT 2
+
+typedef enum PgStat_TableLookupResult
+{
+ NOT_FOUND,
+ FOUND,
+ LOCK_FAILED
+} PgStat_TableLookupResult;
/* ----------
* GUC parameters
@@ -130,31 +112,63 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
-/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-
-/* ----------
- * Local data
- * ----------
- */
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
+#define StatsLock (&StatsShmem->StatsMainLock)
-static time_t last_pgstat_start_time;
+/* Shared stats bootstrap information */
+typedef struct StatsShmemStruct
+{
+ LWLock StatsMainLock; /* lock protecting this struct */
+ dsa_handle stats_dsa_handle; /* DSA handle for stats collector */
+ dshash_table_handle db_hash_handle;
+ dsa_pointer global_stats;
+ dsa_pointer archiver_stats;
+ int refcount;
+} StatsShmemStruct;
-static bool pgStatRunningInCollector = false;
+/*
+ * BgWriter global statistics counters. The name is the remnant from the time
+ * when the stats collector was a dedicate process, which used sockets to send
+ * it.
+ */
+PgStat_MsgBgWriter BgWriterStats = {0};
+
+/* Variables lives for the backend lifetime */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
+static dshash_table *pgStatDBHash = NULL;
+
+
+/* parameter for each type of shared hash */
+static const dshash_parameters dsh_dbparams = {
+ sizeof(Oid),
+ SHARED_DBENT_SIZE,
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+static const dshash_parameters dsh_tblparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatTabEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+static const dshash_parameters dsh_funcparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatFuncEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
/*
* Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
+ * written to shared memory.
*
* NOTE: once allocated, TabStatusArray structures are never moved or deleted
* for the life of the backend. Also, we zero out the t_id fields of the
@@ -189,8 +203,8 @@ typedef struct TabStatHashEntry
static HTAB *pgStatTabHash = NULL;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Backends store per-function info that's waiting to be flushed out to shared
+ * memory in this hash table (indexed by function OID).
*/
static HTAB *pgStatFunctions = NULL;
@@ -200,6 +214,68 @@ static HTAB *pgStatFunctions = NULL;
*/
static bool have_function_stats = false;
+/* common header of snapshot entry in backend snapshot hash */
+typedef struct PgStat_snapshot
+{
+ Oid key;
+ bool negative;
+ void *body; /* end of header part: to keep alignment */
+} PgStat_snapshot;
+
+/* context struct for snapshot_statentry */
+typedef struct pgstat_snapshot_param
+{
+ char *hash_name; /* name of the snapshot hash */
+ int hash_entsize; /* element size of hash entry */
+ dshash_table_handle dsh_handle; /* dsh handle to attach */
+ const dshash_parameters *dsh_params;/* dshash params */
+ HTAB **hash; /* points to variable to hold hash */
+ dshash_table **dshash; /* ditto for dshash */
+} pgstat_snapshot_param;
+
+/*
+ * Backends store various database-wide info that's waiting to be flushed out
+ * to shared memory in these variables.
+ *
+ * checksum_failures is the exception in that it is cluster-wide.
+ */
+typedef struct BackendDBStats
+{
+ int n_conflict_tablespace;
+ int n_conflict_lock;
+ int n_conflict_snapshot;
+ int n_conflict_bufferpin;
+ int n_conflict_startup_deadlock;
+ int n_deadlocks;
+ size_t n_tmpfiles;
+ size_t tmpfilesize;
+ HTAB *checksum_failures;
+} BackendDBStats;
+
+/* Hash entry struct for checksum_failures above */
+typedef struct ChecksumFailureEnt
+{
+ Oid dboid;
+ int count;
+} ChecksumFailureEnt;
+
+static BackendDBStats BeDBStats = {0};
+
+/* macros to check BeDBStats at once */
+#define HAVE_PENDING_CONFLICTS() \
+ (BeDBStats.n_conflict_tablespace > 0 || \
+ BeDBStats.n_conflict_lock > 0 || \
+ BeDBStats.n_conflict_bufferpin > 0 || \
+ BeDBStats.n_conflict_startup_deadlock > 0)
+
+#define HAVE_PENDING_DBSTATS() \
+ (HAVE_PENDING_CONFLICTS() || \
+ BeDBStats.n_deadlocks > 0 || \
+ BeDBStats.n_tmpfiles > 0 || \
+ /* no need to check tmpfilesize */ \
+ BeDBStats.checksum_failures != NULL)
+
+
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
* into PgStat_TableStatus counters until we know if it is going to commit
@@ -235,11 +311,11 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatLocalHash = NULL;
+static bool clear_snapshot = false;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -248,23 +324,35 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Struct for context for pgstat_flush_* functions
+ *
+ * To avoid repeated attach/detch of the same dshash, dshashes once attached
+ * is stored in this structure and moved around multiple calls and multiple
+ * functions. generation here means the value returned by pin_hashes().
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
+typedef struct pgstat_flush_stat_context
+{
+ int shgeneration; /* "generation" of shdb_tabhash below */
+ PgStat_StatDBEntry *shdbentry; /* dbentry for shared tables (oid = 0) */
+ dshash_table *shdb_tabhash; /* tabentry dshash of shared tables */
+
+ int mygeneration; /* "generation" of mydb_tabhash below */
+ PgStat_StatDBEntry *mydbentry; /* dbengry for my database */
+ dshash_table *mydb_tabhash; /* tabentry dshash of my database */
+} pgstat_flush_stat_context;
/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are collected not per database nor per table
+ * basis. shared_* points to shared memroy and snapshot_* are backend
+ * snapshots. Their validity is indicated by global_snapshot_is_valid.
*/
-static List *pending_write_requests = NIL;
-
-/* Signal handler flags */
-static volatile bool need_exit = false;
-static volatile bool got_SIGHUP = false;
+static bool global_snapshot_is_valid = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats snapshot_globalStats;
/*
* Total time charged to functions so far in the current backend.
@@ -278,35 +366,41 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-static void pgstat_exit(SIGNAL_ARGS);
static void pgstat_beshutdown_hook(int code, Datum arg);
-static void pgstat_sighup_handler(SIGNAL_ARGS);
-
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
+static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, int op,
+ PgStat_TableLookupResult *status);
+static PgStat_StatTabEntry *pgstat_get_tab_entry(dshash_table *table,
Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static void pgstat_write_pgStatDBHashfile(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_pgStatDBHashfile(PgStat_StatDBEntry *dbentry);
static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
+static bool pgstat_flush_stat(pgstat_flush_stat_context *cxt, bool nowait);
+static bool pgstat_flush_tabstat(pgstat_flush_stat_context *cxt, bool nowait,
+ PgStat_TableStatus *entry);
+static bool pgstat_flush_funcstats(pgstat_flush_stat_context *cxt, bool nowait);
+static bool pgstat_flush_dbstats(pgstat_flush_stat_context *cxt, bool nowait);
+static bool pgstat_update_tabentry(dshash_table *tabhash,
+ PgStat_TableStatus *stat, bool nowait);
+static void pgstat_update_dbentry(PgStat_StatDBEntry *dbentry,
+ PgStat_TableStatus *stat);
static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
+static void pgstat_remove_useless_entries(const dshash_table_handle dshhandle,
+ const dshash_parameters *dshparams,
+ HTAB *oidtab);
static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
static void pgstat_setup_memcxt(void);
+static void pgstat_flush_recovery_conflict(PgStat_StatDBEntry *dbentry);
+static void pgstat_flush_deadlock(PgStat_StatDBEntry *dbentry);
+static void pgstat_flush_checksum_failure(PgStat_StatDBEntry *dbentry);
+static void pgstat_flush_tempfile(PgStat_StatDBEntry *dbentry);
+static HTAB *create_tabstat_hash(void);
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry_extended(PgStat_StatDBEntry *dbent, Oid funcid);
+static void pgstat_snapshot_global_stats(void);
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
@@ -314,557 +408,491 @@ static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
+/* ------------------------------------------------------------
+ * Local support functions follow
+ * ------------------------------------------------------------
+ */
+static int pin_hashes(PgStat_StatDBEntry *dbentry);
+static void unpin_hashes(PgStat_StatDBEntry *dbentry, int generation);
+static dshash_table *attach_table_hash(PgStat_StatDBEntry *dbent, int gen);
+static dshash_table *attach_function_hash(PgStat_StatDBEntry *dbent, int gen);
+static void reset_dbentry_counters(PgStat_StatDBEntry *dbentry);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute space needed for stats collector's shared memory
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
+ bool found;
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
- }
+ Assert(!found);
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ }
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
+ LWLockInitialize(StatsLock, LWTRANCHE_STATS);
+}
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+/* ----------
+ * pgstat_attach_shared_stats() -
+ *
+ * Attach shared or create stats memory.
+ * ---------
+ */
+static void
+pgstat_attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * Don't use dsm under postmaster, when not tracking counts.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ pgstat_setup_memcxt();
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ if (area)
+ return;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
+ {
+ /* Need to create shared memory area and load saved stats if any. */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatDBHash = dshash_create(area, &dsh_dbparams, 0);
- test_byte++; /* just make sure variable is changed */
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->db_hash_handle = dshash_get_hash_table_handle(pgStatDBHash);
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Load saved data if any. */
+ pgstat_read_statsfiles();
- /* If we get here, we have a working socket */
- break;
+ StatsShmem->refcount = 1;
}
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockRelease(StatsLock);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * If we're not the first process, attach existing shared stats area
+ * outside StatsLock.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (!area)
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* Shared area already exists. Just attach it. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatDBHash = dshash_attach(area, &dsh_dbparams,
+ StatsShmem->db_hash_handle, 0);
+
+ /* Setup local variables */
+ pgStatLocalHash = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
}
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
+ MemoryContextSwitchTo(oldcontext);
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
+ dsa_pin_mapping(area);
+ global_snapshot_is_valid = false;
+}
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
+/* ----------
+ * pgstat_detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and
+ * instructed to write file.
+ * ----------
+ */
+static void
+pgstat_detach_shared_stats(bool write_stats)
+{
+ if (!area || !IsUnderPostmaster)
+ return;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ /* write out the shared stats to file if needed */
+ if (--StatsShmem->refcount < 1)
+ {
+ if (write_stats)
+ pgstat_write_statsfiles();
+
+ /* We're the last process. Invalidate the dsa area handle. */
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+ LWLockRelease(StatsLock);
- return;
+ /*
+ * Detach the area. Automatically destroyed when the last process detached
+ * it.
+ */
+ dsa_detach(area);
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
+ area = NULL;
+ pgStatDBHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatLocalHash = NULL;
+ global_snapshot_is_valid = false;
+}
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+/*
+ * pgstat_reset_all() -
+ *
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
+ */
+void
+pgstat_reset_all(void)
+{
+ /* we must have shared stats attached */
+ Assert (StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
+ /* Startup must be the only user of shared stats */
+ Assert (StatsShmem->refcount == 1);
/*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
+ * We could directly remove files and recreate the shared memory area. But
+ * detach then attach for simplicity.
*/
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ pgstat_detach_shared_stats(false); /* Don't write */
+ pgstat_attach_shared_stats();
}
-/*
- * subroutine for pgstat_reset_all
+/* ------------------------------------------------------------
+ * Public functions used by backends follow
+ *------------------------------------------------------------
*/
-static void
-pgstat_reset_remove_files(const char *directory)
+
+/* ----------
+ * pgstat_report_stat() -
+ *
+ * Must be called by processes that performs DML: tcop/postgres.c, logical
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_STAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_STAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates holded for more than
+ * PGSTAT_STAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
+ * transaction stop time as an approximation of current time.
+ * ----------
+ */
+long
+pgstat_report_stat(bool force)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ TimestampTz now;
+ pgstat_flush_stat_context cxt = {0};
+ bool pending_stats = false;
+ long elapsed;
+ long secs;
+ int usecs;
+
+ /* Don't expend a clock check if nothing to do */
+ if (area == NULL ||
+ ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
+ pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
+ !HAVE_PENDING_DBSTATS() && !have_function_stats))
+ return 0;
+
+ now = GetCurrentTransactionStopTimestamp();
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
+ if (!force)
{
- int nchars;
- Oid tmp_oid;
+ /*
+ * Don't flush stats unless it's the time. Returns time to wait in
+ * milliseconds.
+ */
+ if (now < next_flush)
+ {
+ /* Record the oldest pending update if not yet. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ /* now < next_flush here */
+ return (next_flush - now) / 1000;
+ }
/*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
+ * Don't keep pending updates longer than PGSTAT_STAT_MAX_INTERVAL.
*/
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
+ if (pending_since > 0)
{
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
+ TimestampDifference(pending_since, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs /1000;
+
+ if(elapsed > PGSTAT_STAT_MAX_INTERVAL)
+ force = true;
}
+ }
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
+ /* Flush out table stats */
+ if (pgStatTabList != NULL && !pgstat_flush_stat(&cxt, !force))
+ pending_stats = true;
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ /* Flush out function stats */
+ if (pgStatFunctions != NULL && !pgstat_flush_funcstats(&cxt, !force))
+ pending_stats = true;
+
+ /* Flush out database-wide stats */
+ if (HAVE_PENDING_DBSTATS())
+ {
+ if (!pgstat_flush_dbstats(&cxt, !force))
+ pending_stats = true;
}
- FreeDir(dir);
-}
-/*
- * pgstat_reset_all() -
- *
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
- */
-void
-pgstat_reset_all(void)
-{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
+ /* Unpin dbentry if pinned */
+ if (cxt.mydb_tabhash)
+ {
+ dshash_detach(cxt.mydb_tabhash);
+ unpin_hashes(cxt.mydbentry, cxt.mygeneration);
+ cxt.mydb_tabhash = NULL;
+ cxt.mydbentry = NULL;
+ }
-#ifdef EXEC_BACKEND
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (shared_globalStats->stats_timestamp < now)
+ shared_globalStats->stats_timestamp = now;
+ LWLockRelease(StatsLock);
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
+ /* Record how long we are keepnig pending updats. */
+ if (pending_stats)
+ {
+ /* Preserve the first value */
+ if (pending_since == 0)
+ pending_since = now;
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ /*
+ * It's possible that the retry interval is longer than the limit by
+ * PGSTAT_STAT_MAX_INTERVAL. We don't bother that since it's not so
+ * much.
+ */
+ return PGSTAT_STAT_RETRY_INTERVAL;
+ }
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_STAT_MIN_INTERVAL * 1000;
+ pending_since = 0;
- return postmaster_forkexec(ac, av);
+ return 0;
}
-#endif /* EXEC_BACKEND */
-
/*
- * pgstat_start() -
+ * snapshot_statentry() - Common routine for functions
+ * pgstat_fetch_stat_*entry()
*
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
+ * Returns the pointer to a snapshot of a shared entry for the key or NULL if
+ * not found. Returned snapshots are stable during the current transaction or
+ * until pgstat_clear_snapshot() is called.
*
- * Returns PID of child process, or 0 if fail.
+ * The snapshots are stored in a hash, pointer to which is stored in the
+ * *HTAB variable pointed by cxt->hash. If not created yet, it is created
+ * using hash_name, hash_entsize in cxt.
*
- * Note: if fail, we will be called again from the postmaster main loop.
+ * cxt->dshash points to dshash_table for dbstat entries. If not yet
+ * attached, it is attached using cxt->dsh_handle.
*/
-int
-pgstat_start(void)
+static void *
+snapshot_statentry(pgstat_snapshot_param *cxt, Oid key)
{
- time_t curtime;
- pid_t pgStatPid;
+ PgStat_snapshot *lentry = NULL;
+ size_t keysize = cxt->dsh_params->key_size;
+ size_t dsh_entrysize = cxt->dsh_params->entry_size;
+ bool found;
/*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
+ * We don't want so frequent update of stats snapshot. Keep it at least
+ * for PGSTAT_STAT_MIN_INTERVAL ms. Not postpone but just ignore the cue.
*/
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
+ if (clear_snapshot)
+ {
+ clear_snapshot = false;
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ if (pgStatSnapshotContext &&
+ snapshot_globalStats.stats_timestamp <
+ GetCurrentStatementStartTimestamp() -
+ PGSTAT_STAT_MIN_INTERVAL * 1000)
+ {
+ MemoryContextReset(pgStatSnapshotContext);
+
+ /* Reset variables */
+ global_snapshot_is_valid = false;
+ pgStatSnapshotContext = NULL;
+ pgStatLocalHash = NULL;
+
+ pgstat_setup_memcxt();
+ }
+ }
/*
- * Okay, fork off the collector.
+ * Create new hash, with rather arbitrary initial number of entries since
+ * we don't know how this hash will grow.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
+ if (!*cxt->hash)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
+ HASHCTL ctl;
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
+ /*
+ * Create the hash in the stats context
+ *
+ * The entry is prepended by common header part represented by
+ * PgStat_snapshot.
+ */
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ ctl.keysize = keysize;
+ ctl.entrysize = offsetof(PgStat_snapshot, body) + cxt->hash_entsize;
+ ctl.hcxt = pgStatSnapshotContext;
+ *cxt->hash = hash_create(cxt->hash_name, 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ lentry = hash_search(*cxt->hash, &key, HASH_ENTER, &found);
- PgstatCollectorMain(0, NULL);
- break;
-#endif
+ /*
+ * Refer shared hash if not found in the local hash. We return up-to-date
+ * entries outside a transaction so do the same even if the snapshot is
+ * found.
+ */
+ if (!found || !IsTransactionState())
+ {
+ void *sentry;
- default:
- return (int) pgStatPid;
- }
+ /* attach shared hash if not given, leave it alone for later use */
+ if (!*cxt->dshash)
+ {
+ MemoryContext oldcxt;
- /* shouldn't get here */
- return 0;
-}
+ Assert (cxt->dsh_handle != DSM_HANDLE_INVALID);
+ oldcxt = MemoryContextSwitchTo(pgStatSnapshotContext);
+ *cxt->dshash =
+ dshash_attach(area, cxt->dsh_params, cxt->dsh_handle, NULL);
+ MemoryContextSwitchTo(oldcxt);
+ }
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
-}
+ sentry = dshash_find(*cxt->dshash, &key, false);
-/* ------------------------------------------------------------
- * Public functions used by backends follow
- *------------------------------------------------------------
- */
+ if (sentry)
+ {
+ /*
+ * In transaction state, it is obvious that we should create local
+ * cache entries for consistency. If we are not, we return an
+ * up-to-date entry. Having said that, we need a local copy since
+ * dshash entry must be released immediately. We share the same
+ * local hash entry for the purpose.
+ */
+ memcpy(&lentry->body, sentry, dsh_entrysize);
+ dshash_release_lock(*cxt->dshash, sentry);
+ /* then zero out the local additional space if any */
+ if (dsh_entrysize < cxt->hash_entsize)
+ MemSet((char *)&lentry->body + dsh_entrysize, 0,
+ cxt->hash_entsize - dsh_entrysize);
+ }
-/* ----------
- * pgstat_report_stat() -
+ lentry->negative = !sentry;
+ }
+
+ if (lentry->negative)
+ return NULL;
+
+ return &lentry->body;
+}
+
+/*
+ * pgstat_flush_stat: Flushes table stats out to shared statistics.
*
- * Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
- * transaction stop time as an approximation of current time.
- * ----------
+ * If nowait is true, returns false if required lock was not acquired
+ * immediately. In that case, unapplied table stats updates are left alone in
+ * TabStatusArray to wait for the next chance. cxt holds some dshash related
+ * values that we want to carry around while updating shared stats.
+ *
+ * Returns true if all stats info are flushed. Caller must detach dshashes
+ * stored in cxt after use.
*/
-void
-pgstat_report_stat(bool force)
+static bool
+pgstat_flush_stat(pgstat_flush_stat_context *cxt, bool nowait)
{
- /* we assume this inits to all zeroes: */
static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
- TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
TabStatusArray *tsa;
- int i;
-
- /* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ HTAB *new_tsa_hash = NULL;
+ TabStatusArray *dest_tsa = pgStatTabList;
+ int dest_elem = 0;
+ int i;
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
- now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
+ /* nothing to do, just return */
+ if (pgStatTabHash == NULL)
+ return true;
/*
* Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
+ * entries it points to.
*/
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
+ hash_destroy(pgStatTabHash);
pgStatTabHash = NULL;
/*
* Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * have counts, and try flushing it out to shared stats. We may fail on
+ * some entries in the array. Leaving the entries being packed at the
+ * beginning of the array.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
{
for (i = 0; i < tsa->tsa_used; i++)
{
PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
/* Shouldn't have any pending transaction-dependent counts */
Assert(entry->trans == NULL);
@@ -877,178 +905,352 @@ pgstat_report_stat(bool force)
sizeof(PgStat_TableCounts)) == 0)
continue;
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* try to apply the tab stats */
+ if (!pgstat_flush_tabstat(cxt, nowait, entry))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ /*
+ * Failed. Move it to the beginning in TabStatusArray and
+ * leave it.
+ */
+ TabStatHashEntry *hash_entry;
+ bool found;
+
+ if (new_tsa_hash == NULL)
+ new_tsa_hash = create_tabstat_hash();
+
+ /* Create hash entry for this entry */
+ hash_entry = hash_search(new_tsa_hash, &entry->t_id,
+ HASH_ENTER, &found);
+ Assert(!found);
+
+ /*
+ * Move insertion pointer to the next segment if the segment
+ * is filled up.
+ */
+ if (dest_elem >= TABSTAT_QUANTUM)
+ {
+ Assert(dest_tsa->tsa_next != NULL);
+ dest_tsa = dest_tsa->tsa_next;
+ dest_elem = 0;
+ }
+
+ /*
+ * Pack the entry at the begining of the array. Do nothing if
+ * no need to be moved.
+ */
+ if (tsa != dest_tsa || i != dest_elem)
+ {
+ PgStat_TableStatus *new_entry;
+ new_entry = &dest_tsa->tsa_entries[dest_elem];
+ *new_entry = *entry;
+
+ /* use new_entry as entry hereafter */
+ entry = new_entry;
+ }
+
+ hash_entry->tsa_entry = entry;
+ dest_elem++;
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
}
+ /* zero out unused area of TableStatus */
+ dest_tsa->tsa_used = dest_elem;
+ MemSet(&dest_tsa->tsa_entries[dest_elem], 0,
+ (TABSTAT_QUANTUM - dest_elem) * sizeof(PgStat_TableStatus));
+ while (dest_tsa->tsa_next)
+ {
+ dest_tsa = dest_tsa->tsa_next;
+ MemSet(dest_tsa->tsa_entries, 0,
+ dest_tsa->tsa_used * sizeof(PgStat_TableStatus));
+ dest_tsa->tsa_used = 0;
+ }
+
+ /* and set the new TabStatusArray hash if any */
+ pgStatTabHash = new_tsa_hash;
+
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * We no longer need shared database and table entries, but that for my
+ * database may be used later.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
-
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ if (cxt->shdb_tabhash)
+ {
+ dshash_detach(cxt->shdb_tabhash);
+ unpin_hashes(cxt->shdbentry, cxt->shgeneration);
+ cxt->shdb_tabhash = NULL;
+ cxt->shdbentry = NULL;
+ }
+
+ return pgStatTabHash == NULL;
}
+/* -------
+ * Subroutines for pgstat_flush_stat.
+ * -------
+ */
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * pgstat_flush_tabstat: Flushes a table stats entry.
+ *
+ * If nowait is true, returns false on lock failure. Dshashes for table and
+ * function stats are kept attached in ctx. The caller must detach them after
+ * use.
+ *
+ * Returns true if the entry is flushed out.
*/
-static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+bool
+pgstat_flush_tabstat(pgstat_flush_stat_context *cxt, bool nowait,
+ PgStat_TableStatus *entry)
{
- int n;
- int len;
+ Oid dboid = entry->t_shared ? InvalidOid : MyDatabaseId;
+ int table_mode = PGSTAT_EXCLUSIVE;
+ bool updated = false;
+ dshash_table *tabhash;
+ PgStat_StatDBEntry *dbent;
+ int generation;
+
+ if (nowait)
+ table_mode |= PGSTAT_NOWAIT;
+
+ /* Attach required table hash if not yet. */
+ if ((entry->t_shared ? cxt->shdb_tabhash : cxt->mydb_tabhash) == NULL)
+ {
+ /*
+ * Return if we don't have corresponding dbentry. It would've been
+ * removed.
+ */
+ dbent = pgstat_get_db_entry(dboid, table_mode, NULL);
+ if (!dbent)
+ return false;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ /*
+ * We don't hold lock on the dbentry since it cannot be dropped while
+ * we are working on it.
+ */
+ generation = pin_hashes(dbent);
+ tabhash = attach_table_hash(dbent, generation);
- /*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
- */
- if (OidIsValid(tsmsg->m_databaseid))
- {
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
- pgStatXactCommit = 0;
- pgStatXactRollback = 0;
- pgStatBlockReadTime = 0;
- pgStatBlockWriteTime = 0;
+ if (entry->t_shared)
+ {
+ cxt->shgeneration = generation;
+ cxt->shdbentry = dbent;
+ cxt->shdb_tabhash = tabhash;
+ }
+ else
+ {
+ cxt->mygeneration = generation;
+ cxt->mydbentry = dbent;
+ cxt->mydb_tabhash = tabhash;
+
+ /*
+ * We come here once per database. Take the chance to update
+ * database-wide stats
+ */
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
+ dbent->n_xact_commit += pgStatXactCommit;
+ dbent->n_xact_rollback += pgStatXactRollback;
+ dbent->n_block_read_time += pgStatBlockReadTime;
+ dbent->n_block_write_time += pgStatBlockWriteTime;
+ LWLockRelease(&dbent->lock);
+ pgStatXactCommit = 0;
+ pgStatXactRollback = 0;
+ pgStatBlockReadTime = 0;
+ pgStatBlockWriteTime = 0;
+ }
+ }
+ else if (entry->t_shared)
+ {
+ dbent = cxt->shdbentry;
+ tabhash = cxt->shdb_tabhash;
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ dbent = cxt->mydbentry;
+ tabhash = cxt->mydb_tabhash;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ /*
+ * Local table stats should be applied to both dbentry and tabentry at
+ * once. Update dbentry only if we could update tabentry.
+ */
+ if (pgstat_update_tabentry(tabhash, entry, nowait))
+ {
+ pgstat_update_dbentry(dbent, entry);
+ updated = true;
+ }
+
+ return updated;
}
/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+ * pgstat_flush_funcstats: Flushes function stats.
+ *
+ * If nowait is true, returns false on lock failure. Unapplied local hash
+ * entryis are left alone.
+ *
+ * Returns true if all entries are flushed out.
*/
-static void
-pgstat_send_funcstats(void)
+static bool
+pgstat_flush_funcstats(pgstat_flush_stat_context *cxt, bool nowait)
{
/* we assume this inits to all zeroes: */
static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
+ dshash_table *funchash;
HASH_SEQ_STATUS fstat;
+ PgStat_BackendFunctionEntry *bestat;
+ /* nothing to do, just return */
if (pgStatFunctions == NULL)
- return;
+ return true;
+
+ /* get dbentry into cxt if not yet. */
+ if (cxt->mydbentry == NULL)
+ {
+ int op = PGSTAT_EXCLUSIVE;
+
+ if (nowait)
+ op |= PGSTAT_NOWAIT;
+
+ cxt->mydbentry = pgstat_get_db_entry(MyDatabaseId, op, NULL);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
+ if (cxt->mydbentry == NULL)
+ return false;
+ cxt->mygeneration = pin_hashes(cxt->mydbentry);
+ }
+
+ funchash = attach_function_hash(cxt->mydbentry, cxt->mygeneration);
+ if (funchash == NULL)
+ return false;
+
+ have_function_stats = false;
+
+ /*
+ * Scan through the pgStatFunctions to find functions that actually have
+ * counts, and try flushing it out to shared stats.
+ */
hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ while ((bestat = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
{
- PgStat_FunctionEntry *m_ent;
+ bool found;
+ PgStat_StatFuncEntry *funcent = NULL;
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&bestat->f_counts, &all_zeroes,
sizeof(PgStat_FunctionCounts)) == 0)
continue;
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
+ funcent = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert_extended(funchash, (void *) &(bestat->f_id),
+ &found, nowait);
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
+ /*
+ * We couldn't acquire lock on the required entry. Leave the local
+ * entry alone.
+ */
+ if (!funcent)
{
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
+ have_function_stats = true;
+ continue;
}
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
+ /* Initialize if it's new, or add to it. */
+ if (!found)
+ {
+ funcent->functionid = bestat->f_id;
+ funcent->f_numcalls = bestat->f_counts.f_numcalls;
+ funcent->f_total_time =
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_total_time);
+ funcent->f_self_time =
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_self_time);
+ }
+ else
+ {
+ funcent->f_numcalls += bestat->f_counts.f_numcalls;
+ funcent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_total_time);
+ funcent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_self_time);
+ }
+ dshash_release_lock(funchash, funcent);
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
+ /* reset used counts */
+ MemSet(&bestat->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ }
- have_function_stats = false;
+ return !have_function_stats;
}
+/*
+ * pgstat_flush_dbstats: Flushes out miscellaneous database stats.
+ *
+ * If nowait is true, returns with false on lock failure on dbentry.
+ *
+ * Returns true if all stats are flushed out.
+ */
+static bool
+pgstat_flush_dbstats(pgstat_flush_stat_context *cxt, bool nowait)
+{
+ /* get dbentry if not yet. */
+ if (cxt->mydbentry == NULL)
+ {
+ int op = PGSTAT_EXCLUSIVE;
+ if (nowait)
+ op |= PGSTAT_NOWAIT;
+
+ cxt->mydbentry = pgstat_get_db_entry(MyDatabaseId, op, NULL);
+
+ /* return if lock failed. */
+ if (cxt->mydbentry == NULL)
+ return false;
+
+ /* we use this generation of table /function stats in this turn */
+ cxt->mygeneration = pin_hashes(cxt->mydbentry);
+ }
+
+ LWLockAcquire(&cxt->mydbentry->lock, LW_EXCLUSIVE);
+ if (HAVE_PENDING_CONFLICTS())
+ pgstat_flush_recovery_conflict(cxt->mydbentry);
+ if (BeDBStats.n_deadlocks != 0)
+ pgstat_flush_deadlock(cxt->mydbentry);
+ if (BeDBStats.n_tmpfiles != 0)
+ pgstat_flush_tempfile(cxt->mydbentry);
+ if (BeDBStats.checksum_failures != NULL)
+ pgstat_flush_checksum_failure(cxt->mydbentry);
+ LWLockRelease(&cxt->mydbentry->lock);
+
+ return true;
+}
/* ----------
* pgstat_vacuum_stat() -
*
- * Will tell the collector about objects he can get rid of.
+ * Remove objects we can get rid of.
* ----------
*/
void
pgstat_vacuum_stat(void)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
+ HTAB *oidtab;
+ dshash_seq_status dshstat;
PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
return;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
/*
* Read pg_database and make a list of OIDs of all existing databases
*/
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ oidtab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
/*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
+ * Search the database hash table for dead databases and drop them
+ * from the hash.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+
+ dshash_seq_init(&dshstat, pgStatDBHash, false, true);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&dshstat)) != NULL)
{
Oid dbid = dbentry->databaseid;
@@ -1056,137 +1258,43 @@ pgstat_vacuum_stat(void)
/* the DB entry for shared tables (with InvalidOid) is never dropped */
if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
+ hash_search(oidtab, (void *) &dbid, HASH_FIND, NULL) == NULL)
pgstat_drop_database(dbid);
}
/* Clean up */
- hash_destroy(htab);
+ hash_destroy(oidtab);
/*
* Lookup our own database entry; if not found, nothing more to do.
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_EXCLUSIVE, NULL);
+ if (!dbentry)
return;
/*
* Similarly to above, make a list of all known relations in this DB.
*/
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
+ oidtab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
/*
* Check for all tables listed in stats hashtable if they still exist.
+ * Stats cache is useless here so directly search the shared hash.
*/
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
+ pgstat_remove_useless_entries(dbentry->tables, &dsh_tblparams, oidtab);
/*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
+ * Repeat the above but we needn't bother in the common case where no
+ * function stats are being collected.
*/
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
+ oidtab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
- hash_destroy(htab);
+ pgstat_remove_useless_entries(dbentry->functions, &dsh_funcparams,
+ oidtab);
}
+ dshash_release_lock(pgStatDBHash, dbentry);
}
@@ -1240,66 +1348,99 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
return htab;
}
-
-/* ----------
- * pgstat_drop_database() -
+/*
+ * pgstat_remove_useless_entries - Remove useless entries from per
+ * table/function dshashes.
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Scan the dshash specified by dshhandle removing entries that are not in
+ * oidtab. oidtab is destroyed before returning.
*/
void
-pgstat_drop_database(Oid databaseid)
+pgstat_remove_useless_entries(const dshash_table_handle dshhandle,
+ const dshash_parameters *dshparams,
+ HTAB *oidtab)
{
- PgStat_MsgDropdb msg;
+ dshash_table *dshtable;
+ dshash_seq_status dshstat;
+ void *ent;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ dshtable = dshash_attach(area, dshparams, dshhandle, 0);
+ dshash_seq_init(&dshstat, dshtable, false, true);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* The first member of the entries must be Oid */
+ if (hash_search(oidtab, ent, HASH_FIND, NULL) != NULL)
+ continue;
+ /* Not there, so purge this entry */
+ dshash_delete_entry(dshtable, ent);
+ }
+ dshash_detach(dshtable);
+ hash_destroy(oidtab);
+}
/* ----------
- * pgstat_drop_relation() -
+ * pgstat_drop_database() -
*
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
+ * Remove entry for the database that we just dropped.
*
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
+ * If some stats are flushed after this, this entry will be re-created but we
+ * will still clean the dead DB eventually via future invocations of
+ * pgstat_vacuum_stat().
* ----------
*/
-#ifdef NOT_USED
void
-pgstat_drop_relation(Oid relid)
+pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgTabpurge msg;
- int len;
+ PgStat_StatDBEntry *dbentry;
+
+ Assert (OidIsValid(databaseid));
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatDBHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
+ /*
+ * Lookup the database in the hashtable with exclusive lock.
+ */
+ dbentry = pgstat_get_db_entry(databaseid, PGSTAT_EXCLUSIVE, NULL);
+
+ /*
+ * If found, remove it.
+ */
+ if (dbentry)
+ {
+ /* LWLock is needed to rewrite */
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+
+ /* No one is using tables/functions in this dbentry */
+ Assert(dbentry->refcnt == 0);
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
+ /* Remove table/function stats dshash first. */
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(tbl);
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(tbl);
+ }
+ LWLockRelease(&dbentry->lock);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ dshash_delete_entry(pgStatDBHash, (void *)dbentry);
+ }
}
-#endif /* NOT_USED */
-
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1308,20 +1449,32 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupResult status;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!pgStatDBHash)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Lookup the database in the hashtable. Nothing to do if not there.
+ */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_EXCLUSIVE, &status);
+
+ if (!dbentry)
+ return;
+
+ /* This database is active, safe to release the lock immediately. */
+ dshash_release_lock(pgStatDBHash, dbentry);
+
+ /* Reset database-level stats. */
+ reset_dbentry_counters(dbentry);
+
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1330,29 +1483,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ /* Reset the bgwriter statistics for the cluster. */
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1361,18 +1522,43 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
+ int generation;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_EXCLUSIVE, NULL);
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!dbentry)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* This database is active, safe to release the lock immediately. */
+ generation = pin_hashes(dbentry);
- pgstat_send(&msg, sizeof(msg));
-}
+ /* Set the reset timestamp for the whole database */
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&dbentry->lock);
+
+ /* Remove object if it exists, ignore if not */
+ if (type == RESET_TABLE)
+ {
+ dshash_table *t = attach_table_hash(dbentry, generation);
+ dshash_delete_key(t, (void *) &objoid);
+ dshash_detach(t);
+ }
+
+ if (type == RESET_FUNCTION)
+ {
+ dshash_table *t = attach_function_hash(dbentry, generation);
+ if (t)
+ {
+ dshash_delete_key(t, (void *) &objoid);
+ dshash_detach(t);
+ }
+ }
+ unpin_hashes(dbentry, generation);
+}
/* ----------
* pgstat_report_autovac() -
@@ -1385,48 +1571,81 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ /*
+ * Store the last autovacuum time in the database's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_EXCLUSIVE, NULL);
+ dshash_release_lock(pgStatDBHash, dbentry);
+
+ ts = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->last_autovac_time = ts;
+ LWLockRelease(&dbentry->lock);
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
+ int generation;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hash table entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_EXCLUSIVE, NULL);
+ generation = pin_hashes(dbentry);
+ table = attach_table_hash(dbentry, generation);
+
+ tabentry = pgstat_get_tab_entry(table, tableoid, true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->vacuum_count++;
+ }
+ dshash_release_lock(table, tabentry);
+
+ dshash_detach(table);
+ unpin_hashes(dbentry, generation);
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1437,9 +1656,14 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
+ int generation;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1468,78 +1692,153 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_EXCLUSIVE, NULL);
+ generation = pin_hashes(dbentry);
+ table = attach_table_hash(dbentry, generation);
+ tabentry = pgstat_get_tab_entry(table, RelationGetRelid(rel), true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+ dshash_release_lock(table, tabentry);
+
+ dshash_detach(table);
+ unpin_hashes(dbentry, generation);
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupResult status;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ BeDBStats.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ BeDBStats.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ BeDBStats.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ BeDBStats.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ BeDBStats.n_conflict_startup_deadlock++;
+ break;
+ }
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_EXCLUSIVE | PGSTAT_NOWAIT,
+ &status);
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ if (status == LOCK_FAILED)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ /* We had a chance to flush immediately */
+ pgstat_flush_recovery_conflict(dbentry);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
+}
+
+/*
+ * flush recovery conflict stats
+ */
+static void
+pgstat_flush_recovery_conflict(PgStat_StatDBEntry *dbentry)
+{
+ dbentry->n_conflict_tablespace += BeDBStats.n_conflict_tablespace;
+ dbentry->n_conflict_lock += BeDBStats.n_conflict_lock;
+ dbentry->n_conflict_snapshot += BeDBStats.n_conflict_snapshot;
+ dbentry->n_conflict_bufferpin += BeDBStats.n_conflict_bufferpin;
+ dbentry->n_conflict_startup_deadlock += BeDBStats.n_conflict_startup_deadlock;
+
+ BeDBStats.n_conflict_tablespace = 0;
+ BeDBStats.n_conflict_lock = 0;
+ BeDBStats.n_conflict_snapshot = 0;
+ BeDBStats.n_conflict_bufferpin = 0;
+ BeDBStats.n_conflict_startup_deadlock = 0;
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupResult status;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
+ BeDBStats.n_deadlocks++;
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_EXCLUSIVE | PGSTAT_NOWAIT,
+ &status);
-/* --------
- * pgstat_report_checksum_failures_in_db() -
- *
- * Tell the collector about one or more checksum failures.
- * --------
- */
-void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
-{
- PgStat_MsgChecksumFailure msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ if (status == LOCK_FAILED)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
+ dshash_release_lock(pgStatDBHash, dbentry);
+}
- pgstat_send(&msg, sizeof(msg));
+/*
+ * flush dead lock stats
+ */
+static void
+pgstat_flush_deadlock(PgStat_StatDBEntry *dbentry)
+{
+ dbentry->n_deadlocks += BeDBStats.n_deadlocks;
+ BeDBStats.n_deadlocks = 0;
}
/* --------
@@ -1557,60 +1856,153 @@ pgstat_report_checksum_failure(void)
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupResult status;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ if (filesize > 0) /* Is there a case where filesize is really 0? */
+ {
+ BeDBStats.tmpfilesize += filesize; /* needs check overflow */
+ BeDBStats.n_tmpfiles++;
+ }
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ if (BeDBStats.n_tmpfiles == 0)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_EXCLUSIVE | PGSTAT_NOWAIT,
+ &status);
+
+ if (status == LOCK_FAILED)
+ return;
+
+ /* We had a chance to flush immediately */
+ pgstat_flush_tempfile(dbentry);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
}
+/*
+ * flush temporary file stats
+ */
+static void
+pgstat_flush_tempfile(PgStat_StatDBEntry *dbentry)
+{
-/* ----------
- * pgstat_ping() -
+ dbentry->n_temp_bytes += BeDBStats.tmpfilesize;
+ dbentry->n_temp_files += BeDBStats.n_tmpfiles;
+ BeDBStats.tmpfilesize = 0;
+ BeDBStats.n_tmpfiles = 0;
+}
+
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Send some junk data to the collector to increase traffic.
- * ----------
+ * Tell the collector about one or more checksum failures.
+ * --------
*/
void
-pgstat_ping(void)
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgDummy msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupResult status;
+ ChecksumFailureEnt *failent = NULL;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ if (BeDBStats.checksum_failures != NULL)
+ {
+ failent = hash_search(BeDBStats.checksum_failures, &dboid,
+ HASH_FIND, NULL);
+ if (failent)
+ failurecount += failent->count;
+ }
+
+ if (failurecount == 0)
+ return;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_EXCLUSIVE | PGSTAT_NOWAIT,
+ &status);
+
+ if (status == LOCK_FAILED)
+ {
+ if (!failent)
+ {
+ if (!BeDBStats.checksum_failures)
+ {
+ HASHCTL ctl;
- if (pgStatSock == PGINVALID_SOCKET)
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(ChecksumFailureEnt);
+ BeDBStats.checksum_failures =
+ hash_create("pgstat checksum failure count hash",
+ 32, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ failent = hash_search(BeDBStats.checksum_failures,
+ &dboid, HASH_ENTER, NULL);
+ }
+
+ failent->count = failurecount;
return;
+ }
+
+ /* We have a chance to flush immediately */
+ dbentry->n_checksum_failures += failurecount;
+ BeDBStats.checksum_failures = NULL;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dshash_release_lock(pgStatDBHash, dbentry);
}
-/* ----------
- * pgstat_send_inquiry() -
- *
- * Notify collector that we need fresh data.
- * ----------
+/*
+ * flush checkpoint failure count for all databases
*/
static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
+pgstat_flush_checksum_failure(PgStat_StatDBEntry *dbentry)
{
- PgStat_MsgInquiry msg;
+ HASH_SEQ_STATUS stat;
+ ChecksumFailureEnt *ent;
+ bool release_dbent;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
+ if (BeDBStats.checksum_failures == NULL)
+ return;
+
+ hash_seq_init(&stat, BeDBStats.checksum_failures);
+ while ((ent = (ChecksumFailureEnt *) hash_seq_search(&stat)) != NULL)
+ {
+ release_dbent = false;
+
+ if (dbentry->databaseid != ent->dboid)
+ {
+ dbentry = pgstat_get_db_entry(ent->dboid,
+ PGSTAT_EXCLUSIVE, NULL);
+ if (!dbentry)
+ continue;
+
+ release_dbent = true;
+ }
+
+ dbentry->n_checksum_failures += ent->count;
+ if (release_dbent)
+ dshash_release_lock(pgStatDBHash, dbentry);
+ }
+
+ hash_destroy(BeDBStats.checksum_failures);
+ BeDBStats.checksum_failures = NULL;
+}
/*
* Initialize function call usage data.
@@ -1762,7 +2154,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1781,6 +2174,24 @@ pgstat_initstats(Relation rel)
rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
+/*
+ * create_tabstat_hash - create local hash as transactional storage
+ */
+static HTAB *
+create_tabstat_hash(void)
+{
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(TabStatHashEntry);
+
+ return hash_create("pgstat TabStatusArray lookup hash table",
+ TABSTAT_QUANTUM,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+}
+
/*
* get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
*/
@@ -1796,18 +2207,7 @@ get_tabstat_entry(Oid rel_id, bool isshared)
* Create hash table if we don't have it already.
*/
if (pgStatTabHash == NULL)
- {
- HASHCTL ctl;
-
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
- }
+ pgStatTabHash = create_tabstat_hash();
/*
* Find an entry or create a new one.
@@ -2420,30 +2820,33 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
- * ----------
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ /* context for snapshot_statentry */
+ static pgstat_snapshot_param param =
+ {
+ .hash_name = "local database stats hash",
+ .hash_entsize = sizeof(PgStat_StatDBEntry),
+ .dsh_handle = DSM_HANDLE_INVALID, /* already attached */
+ .dsh_params = &dsh_dbparams,
+ .hash = &pgStatLocalHash,
+ .dshash = &pgStatDBHash
+ };
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *) snapshot_statentry(¶m, dbid);
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
@@ -2456,51 +2859,66 @@ pgstat_fetch_stat_dbentry(Oid dbid)
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* Lookup our database, then look in its table hash table. */
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
+ if (dbentry == NULL)
+ return NULL;
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_extended(dbentry, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ dbentry = pgstat_fetch_stat_dbentry(InvalidOid);
+ if (dbentry == NULL)
+ return NULL;
+
+ tabentry = pgstat_fetch_stat_tabentry_extended(dbentry, relid);
+ if (tabentry != NULL)
+ return tabentry;
return NULL;
}
+/* ----------
+ * pgstat_fetch_stat_tabentry_extended() -
+ *
+ * Find table stats entry on backends. The returned entries are cached until
+ * transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_extended(PgStat_StatDBEntry *dbent, Oid reloid)
+{
+ /* context for snapshot_statentry */
+ static pgstat_snapshot_param param =
+ {
+ .hash_name = "table stats snapshot hash",
+ .hash_entsize = sizeof(PgStat_StatTabEntry),
+ .dsh_handle = DSM_HANDLE_INVALID,
+ .dsh_params = &dsh_tblparams,
+ .hash = NULL,
+ .dshash = NULL
+ };
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* set target shared hash */
+ param.dsh_handle = dbent->tables;
+
+ /* tell snapshot_statentry what variables to use */
+ param.hash = &dbent->snapshot_tables;
+ param.dshash = &dbent->dshash_tables;
+
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(¶m, reloid);
+}
+
/* ----------
* pgstat_fetch_stat_funcentry() -
@@ -2515,21 +2933,90 @@ pgstat_fetch_stat_funcentry(Oid func_id)
PgStat_StatDBEntry *dbentry;
PgStat_StatFuncEntry *funcentry = NULL;
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
+ /* Lookup our database, then find the requested function */
dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
+ if (dbentry == NULL)
+ return NULL;
+
+ funcentry = pgstat_fetch_stat_funcentry_extended(dbentry, func_id);
return funcentry;
}
+/* ----------
+ * pgstat_fetch_stat_funcentry_extended() -
+ *
+ * Find function stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ *
+ * dbent is type of (PgStat_StatDBEntry *) but it's body must be an
+ * PgSTat_StatDBEntry returned from pgstat_fetch_stat_dbentry().
+ */
+static PgStat_StatFuncEntry *
+pgstat_fetch_stat_funcentry_extended(PgStat_StatDBEntry *dbent, Oid funcid)
+{
+ /* context for snapshot_statentry */
+ static pgstat_snapshot_param param =
+ {
+ .hash_name = "function stats snapshot hash",
+ .hash_entsize = sizeof(PgStat_StatFuncEntry),
+ .dsh_handle = DSM_HANDLE_INVALID,
+ .dsh_params = &dsh_funcparams,
+ .hash = NULL,
+ .dshash = NULL
+ };
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ if (dbent->functions == DSM_HANDLE_INVALID)
+ return NULL;
+
+ /* set target shared hash */
+ param.dsh_handle = dbent->functions;
+
+ /* tell snapshot_statentry what variables to use */
+ param.hash = &dbent->snapshot_functions;
+ param.dshash = &dbent->dshash_functions;
+
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(¶m, funcid);
+}
+
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+
+ pgstat_attach_shared_stats();
+
+ /* Nothing to do if already done */
+ if (global_snapshot_is_valid)
+ return;
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ memcpy(&snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ LWLockRelease(StatsLock);
+
+ global_snapshot_is_valid = true;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
+}
/* ----------
* pgstat_fetch_stat_beentry() -
@@ -2601,9 +3088,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return &snapshot_archiverStats;
}
@@ -2618,9 +3106,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return &snapshot_globalStats;
}
@@ -2834,8 +3323,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutodwn */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -2933,7 +3422,7 @@ pgstat_bestart(void)
lbeentry.st_backendType = B_STARTUP;
break;
case ArchiverProcess:
- beentry->st_backendType = B_ARCHIVER;
+ lbeentry.st_backendType = B_ARCHIVER;
break;
case BgWriterProcess:
lbeentry.st_backendType = B_BG_WRITER;
@@ -3069,6 +3558,10 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+
+ /* attach shared database stats area */
+ pgstat_attach_shared_stats();
}
/*
@@ -3104,6 +3597,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ pgstat_detach_shared_stats(true);
}
@@ -3364,7 +3859,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3659,9 +4155,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_ALL:
event_name = "RecoveryWalAll";
break;
@@ -4321,75 +4814,43 @@ pgstat_get_backend_desc(BackendType backendType)
* ------------------------------------------------------------
*/
-
/* ----------
- * pgstat_setheader() -
+ * pgstat_send_archiver() -
*
- * Set common header fields in a statistics message
+ * Report archiver statistics
* ----------
*/
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
+void
+pgstat_send_archiver(const char *xlog, bool failed)
{
- hdr->m_type = mtype;
-}
+ TimestampTz now = GetCurrentTimestamp();
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
- * ----------
- */
-void
-pgstat_send_archiver(const char *xlog, bool failed)
-{
- PgStat_MsgArchiver msg;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
-}
+ if (failed)
+ {
+ /* Failed archival attempt */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ /* Successful archival operation */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+}
/* ----------
* pgstat_send_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
@@ -4398,6 +4859,8 @@ pgstat_send_bgwriter(void)
/* We assume this initializes to zeroes */
static const PgStat_MsgBgWriter all_zeroes;
+ PgStat_MsgBgWriter *s = &BgWriterStats;
+
/*
* This function can be called even if nothing at all has happened. In
* this case, avoid sending a completely empty message to the stats
@@ -4406,11 +4869,18 @@ pgstat_send_bgwriter(void)
if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += s->m_timed_checkpoints;
+ shared_globalStats->requested_checkpoints += s->m_requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += s->m_checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += s->m_checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += s->m_buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += s->m_buf_written_clean;
+ shared_globalStats->maxwritten_clean += s->m_maxwritten_clean;
+ shared_globalStats->buf_written_backend += s->m_buf_written_backend;
+ shared_globalStats->buf_fsync_backend += s->m_buf_fsync_backend;
+ shared_globalStats->buf_alloc += s->m_buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4419,305 +4889,164 @@ pgstat_send_bgwriter(void)
}
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
+/*
+ * Pin and Unpin dbentry.
*
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
+ * To keep less memory usage, and for speed, counters are by recreation of
+ * dshash instead of removing entries one-by-one keeping whole-dshash lock. On
+ * the other hand dshash cannot be destroyed until all referrers have gone. As
+ * the result, other backend may be kept waiting the counter reset for not a
+ * short time. We isolate the hashes under destruction as another generation,
+ * which means no longer used but cannot be removed yet.
+
+ * When we start accessing hashes on a dbentry, call pin_hashes() and acquire
+ * the current "generation". Unlock removes the older generation's hashes when
+ * all refers have gone.
*/
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+static int
+pin_hashes(PgStat_StatDBEntry *dbentry)
{
- int len;
- PgStat_Msg msg;
- int wr;
+ int generation;
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, pgstat_sighup_handler);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, pgstat_exit);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->refcnt++;
+ generation = dbentry->generation;
+ LWLockRelease(&dbentry->lock);
- /*
- * Identify myself via ps
- */
- init_ps_display("stats collector", "", "", "");
+ dshash_release_lock(pgStatDBHash, dbentry);
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
+ return generation;
+}
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do recognize got_SIGHUP inside
- * the inner loop, which means that such interrupts will get serviced but
- * the latch won't get cleared until next time there is a break in the
- * action.
- */
- for (;;)
+/*
+ * Unpin hashes in dbentry. If given generation is isolated, destroy it after
+ * all referrers has gone. Otherwise just decrease reference count then return.
+ */
+static void
+unpin_hashes(PgStat_StatDBEntry *dbentry, int generation)
+{
+ dshash_table *tables;
+ dshash_table *funcs = NULL;
+
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+
+ /* using current generation, just decrease refcount */
+ if (dbentry->generation == generation)
{
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
+ dbentry->refcnt--;
+ LWLockRelease(&dbentry->lock);
+ return;
+ }
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (need_exit)
- break;
+ /*
+ * It is isolated, waiting for all referrers to end.
+ */
+ Assert(dbentry->generation == generation + 1);
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * need_exit becomes set.
- */
- while (!need_exit)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (got_SIGHUP)
- {
- got_SIGHUP = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
+ if (--dbentry->prev_refcnt > 0)
+ {
+ LWLockRelease(&dbentry->lock);
+ return;
+ }
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
+ /* no referrer remains, remove the hashes */
+ tables = dshash_attach(area, &dsh_tblparams, dbentry->prev_tables, 0);
+ if (dbentry->prev_functions != DSM_HANDLE_INVALID)
+ funcs = dshash_attach(area, &dsh_funcparams,
+ dbentry->prev_functions, 0);
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
+ dbentry->prev_tables = DSM_HANDLE_INVALID;
+ dbentry->prev_functions = DSM_HANDLE_INVALID;
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
+ /* release the entry immediately */
+ LWLockRelease(&dbentry->lock);
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
+ dshash_destroy(tables);
+ if (funcs)
+ dshash_destroy(funcs);
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
+ return;
+}
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
+/*
+ * attach and return the specified generation of table hash
+ * Returns NULL on lock failure.
+ */
+static dshash_table *
+attach_table_hash(PgStat_StatDBEntry *dbent, int gen)
+{
+ dshash_table *ret;
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(
- &msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(
- &msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(
- &msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(
- &msg.msg_checksumfailure,
- len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
+ if (dbent->generation == gen)
+ ret = dshash_attach(area, &dsh_tblparams, dbent->tables, 0);
+ else
+ {
+ Assert (dbent->generation == gen + 1);
+ Assert (dbent->prev_tables != DSM_HANDLE_INVALID);
+ ret = dshash_attach(area, &dsh_tblparams, dbent->prev_tables, 0);
+ }
+ LWLockRelease(&dbent->lock);
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
+ return ret;
+}
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
+/* attach and return the specified generation of function hash */
+static dshash_table *
+attach_function_hash(PgStat_StatDBEntry *dbent, int gen)
+{
+ dshash_table *ret = NULL;
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
- exit(0);
-}
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
+ if (dbent->generation == gen)
+ {
+ if (dbent->functions == DSM_HANDLE_INVALID)
+ {
+ dshash_table *funchash =
+ dshash_create(area, &dsh_funcparams, 0);
+ dbent->functions = dshash_get_hash_table_handle(funchash);
-/* SIGQUIT signal handler for collector process */
-static void
-pgstat_exit(SIGNAL_ARGS)
-{
- int save_errno = errno;
+ ret = funchash;
+ }
+ else
+ ret = dshash_attach(area, &dsh_funcparams, dbent->functions, 0);
+ }
+ /* don't bother creating useless hash */
- need_exit = true;
- SetLatch(MyLatch);
+ LWLockRelease(&dbent->lock);
- errno = save_errno;
+ return ret;
}
-/* SIGHUP handler for collector process */
static void
-pgstat_sighup_handler(SIGNAL_ARGS)
+init_dbentry(PgStat_StatDBEntry *dbentry)
{
- int save_errno = errno;
-
- got_SIGHUP = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
+ LWLockInitialize(&dbentry->lock, LWTRANCHE_STATS);
+ dbentry->generation = 0;
+ dbentry->refcnt = 0;
+ dbentry->prev_refcnt = 0;
+ dbentry->tables = DSM_HANDLE_INVALID;
+ dbentry->prev_tables = DSM_HANDLE_INVALID;
+ dbentry->functions = DSM_HANDLE_INVALID;
+ dbentry->prev_functions = DSM_HANDLE_INVALID;
}
/*
* Subroutine to clear stats in a database entry
*
- * Tables and functions hashes are initialized to empty.
+ * Reset all counters in the dbentry. Tables and functions dshashes are
+ * destroyed. If any backend is pinning this dbentry, the current dshashes
+ * are stashed out to the previous "generation" to wait for all accessors are
+ * gone. If the previous generation is already occupied, the current dshashes
+ * are so fresh that they doesn't need to be cleared.
*/
static void
reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
{
- HASHCTL hash_ctl;
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
dbentry->n_xact_commit = 0;
dbentry->n_xact_rollback = 0;
@@ -4742,130 +5071,118 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
dbentry->n_block_read_time = 0;
dbentry->n_block_write_time = 0;
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
+ if (dbentry->refcnt == 0)
+ {
+ /*
+ * No one is referring to the current hash. It's very costly to remove
+ * entries in dshash individually so just destroy the whole. If
+ * someone pined this entry just after, pin_hashes() returns the
+ * current generation and attach will happen after the following
+ * LWLock released.
+ */
+ dshash_table *tbl;
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
-{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(tbl);
+ dbentry->tables = DSM_HANDLE_INVALID;
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ tbl = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(tbl);
+ dbentry->functions = DSM_HANDLE_INVALID;
+ }
+ }
+ else if (dbentry->prev_refcnt == 0)
+ {
+ /*
+ * Someone is still referring to the current hash and previous slot is
+ * vacant. Stash out the current hash to the previous slot.
+ */
+ dbentry->prev_refcnt = dbentry->refcnt;
+ dbentry->prev_tables = dbentry->tables;
+ dbentry->prev_functions = dbentry->functions;
+ dbentry->refcnt = 0;
+ dbentry->tables = DSM_HANDLE_INVALID;
+ dbentry->functions = DSM_HANDLE_INVALID;
+ dbentry->generation++;
+ }
+ else
+ {
+ Assert(dbentry->prev_refcnt > 0 && dbentry->refcnt > 0);
+ /*
+ * If we get here, we just have got another reset request and the old
+ * hashes are waiting to all referrers to be released. It must be
+ * quite a short time so we can just ignore this request.
+ *
+ * As the side effect, the resetter can see non-zero values before
+ * anyone updates them but it's not distinctive with someone updated
+ * them before reading.
+ */
+ }
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
+ /* Create new table hash if not exists */
+ if (dbentry->tables == DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables = dshash_get_hash_table_handle(tbl);
+ dshash_detach(tbl);
+ }
- if (!create && !found)
- return NULL;
+ /* Create new function hash if not exists and needed. */
+ if (dbentry->functions == DSM_HANDLE_INVALID &&
+ pgstat_track_functions != TRACK_FUNC_OFF)
+ {
+ dshash_table *tbl = dshash_create(area, &dsh_funcparams, 0);
+ dbentry->functions = dshash_get_hash_table_handle(tbl);
+ dshash_detach(tbl);
+ }
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
+ dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- return result;
+ LWLockRelease(&dbentry->lock);
}
-
/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+ * Create the filename for a DB stat file; filename is output parameter points
+ * to a character buffer of length len.
*/
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
+static void
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
+ int printed;
- return result;
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
}
-
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
+ dshash_seq_status hstat;
PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
+
elog(DEBUG2, "writing stats file \"%s\"", statfile);
/*
@@ -4884,7 +5201,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4896,39 +5213,37 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_init(&hstat, pgStatDBHash, false, false);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&hstat)) != NULL)
{
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_pgStatDBHashfile(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
* pointers, since they're of no use to any other process.
*/
fputc('D', fpout);
- rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
+ rc = fwrite(dbentry,
+ offsetof(PgStat_StatDBEntry, generation), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
@@ -4964,53 +5279,18 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
-}
-
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
}
/* ----------
- * pgstat_write_db_statsfile() -
+ * pgstat_write_pgStatDBHashfile() -
* Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_pgStatDBHashfile(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
+ dshash_seq_status tstat;
+ dshash_seq_status fstat;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
FILE *fpout;
@@ -5019,9 +5299,10 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
int rc;
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
+ dshash_table *tbl;
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -5048,23 +5329,30 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&tstat, tbl, false, false);
+ while ((tabentry = (PgStat_StatTabEntry *) dshash_seq_next(&tstat)) != NULL)
{
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ dshash_detach(tbl);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ tbl = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_seq_init(&fstat, tbl, false, false);
+ while ((funcentry = (PgStat_StatFuncEntry *) dshash_seq_next(&fstat)) != NULL)
+ {
+ fputc('F', fpout);
+ rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ dshash_detach(tbl);
}
/*
@@ -5099,76 +5387,37 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
}
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
- *
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
+ * Reads in existing statistics collector files into the shared stats hash.
*
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
+
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
- * The tables will live in pgStatLocalContext.
+ * Set the current timestamp (will be kept only in case we can't load an
+ * existing statsfile).
*/
- pgstat_setup_memcxt();
-
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
-
- /*
- * Set the current timestamp (will be kept only in case we can't load an
- * existing statsfile).
- */
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
@@ -5182,11 +5431,11 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5195,7 +5444,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5203,32 +5452,24 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
@@ -5245,10 +5486,10 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
* follows.
*/
case 'D':
- if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
+ if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, generation),
+ fpin) != offsetof(PgStat_StatDBEntry, generation))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5257,76 +5498,36 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+ dbentry = (PgStat_StatDBEntry *)
+ dshash_find_or_insert(pgStatDBHash, (void *) &dbbuf.databaseid,
+ &found);
+
+ /* don't allow duplicate dbentries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(pgStatDBHash, dbentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ /* initialize the new shared entry */
+ init_dbentry(dbentry);
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, generation));
+ /* Read the data from the database-specific file. */
+ pgstat_read_pgStatDBHashfile(dbentry);
+ dshash_release_lock(pgStatDBHash, dbentry);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5336,45 +5537,35 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
/* ----------
- * pgstat_read_db_statsfile() -
+ * pgstat_read_pgStatDBHashfile() -
*
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_pgStatDBHashfile(PgStat_StatDBEntry *dbentry)
{
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
FILE *fpin;
int32 format_id;
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
@@ -5388,7 +5579,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5401,14 +5592,14 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing statistics file. Read it and put all the hashtable
+ * entries into place.
*/
for (;;)
{
@@ -5421,31 +5612,35 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if table data not wanted.
- */
if (tabhash == NULL)
- break;
+ {
+ tabhash = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables =
+ dshash_get_hash_table_handle(tabhash);
+ }
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(tabhash,
+ (void *) &tabbuf.tableid, &found);
+ /* don't allow duplicate entries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(tabhash, tabentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
memcpy(tabentry, &tabbuf, sizeof(tabbuf));
+ dshash_release_lock(tabhash, tabentry);
break;
/*
@@ -5455,31 +5650,34 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if function data not wanted.
- */
if (funchash == NULL)
- break;
+ {
+ funchash = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->functions =
+ dshash_get_hash_table_handle(funchash);
+ }
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ funcentry = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert(funchash,
+ (void *) &funcbuf.functionid, &found);
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(funchash, funcentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
memcpy(funcentry, &funcbuf, sizeof(funcbuf));
+ dshash_release_lock(funchash, funcentry);
break;
/*
@@ -5489,7 +5687,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5497,295 +5695,39 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
-
/* ----------
* pgstat_setup_memcxt() -
*
- * Create pgStatLocalContext, if not already done.
+ * Create pgStatLocalContext and pgStatSnapshotContext, if not already done.
* ----------
*/
static void
pgstat_setup_memcxt(void)
{
if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+
+ if (!pgStatSnapshotContext)
+ pgStatSnapshotContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Database statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
}
-
/* ----------
* pgstat_clear_snapshot() -
*
@@ -5801,739 +5743,223 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
+ }
+ if (pgStatSnapshotContext)
+ clear_snapshot = true;
+}
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
+static bool
+pgstat_update_tabentry(dshash_table *tabhash, PgStat_TableStatus *stat,
+ bool nowait)
{
- PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ bool found;
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
+ if (tabhash == NULL)
+ return false;
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert_extended(tabhash, (void *) &(stat->t_id),
+ &found, nowait);
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
+ /* failed to acquire lock */
+ if (tabentry == NULL)
+ return false;
+
+ if (!found)
{
/*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
+ * If it's a new table entry, initialize counters to the values we
+ * just got.
*/
+ tabentry->numscans = stat->t_counts.t_numscans;
+ tabentry->tuples_returned = stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched = stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted = stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated = stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted = stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated = stat->t_counts.t_tuples_hot_updated;
+ tabentry->n_live_tuples = stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples = stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze = stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched = stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit = stat->t_counts.t_blocks_hit;
+
+ tabentry->vacuum_timestamp = 0;
+ tabentry->vacuum_count = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_count = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->analyze_count = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ tabentry->autovac_analyze_count = 0;
}
- else if (msg->clock_time < dbentry->stats_timestamp)
+ else
{
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ tabentry->numscans += stat->t_counts.t_numscans;
+ tabentry->tuples_returned += stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched += stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted += stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated += stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted += stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated += stat->t_counts.t_tuples_hot_updated;
+ /* If table was truncated, first reset the live/dead counters */
+ if (stat->t_counts.t_truncated)
{
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
+ tabentry->n_live_tuples = 0;
+ tabentry->n_dead_tuples = 0;
}
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
+ tabentry->n_live_tuples += stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples += stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze += stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched += stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit += stat->t_counts.t_blocks_hit;
}
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
+ dshash_release_lock(tabhash, tabentry);
+
+ return true;
+}
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
+pgstat_update_dbentry(PgStat_StatDBEntry *dbentry, PgStat_TableStatus *stat)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
/*
- * Update database-wide stats.
+ * Add per-table stats to the per-database entry, too.
*/
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->n_tuples_returned += stat->t_counts.t_tuples_returned;
+ dbentry->n_tuples_fetched += stat->t_counts.t_tuples_fetched;
+ dbentry->n_tuples_inserted += stat->t_counts.t_tuples_inserted;
+ dbentry->n_tuples_updated += stat->t_counts.t_tuples_updated;
+ dbentry->n_tuples_deleted += stat->t_counts.t_tuples_deleted;
+ dbentry->n_blocks_fetched += stat->t_counts.t_blocks_fetched;
+ dbentry->n_blocks_hit += stat->t_counts.t_blocks_hit;
+ LWLockRelease(&dbentry->lock);
+}
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
+/*
+ * Lookup shared stats hash table for the specified database. Returns NULL
+ * when PGSTAT_NOWAIT and required lock cannot be acquired.
+ */
+static PgStat_StatDBEntry *
+pgstat_get_db_entry(Oid databaseid, int op, PgStat_TableLookupResult *status)
+{
+ PgStat_StatDBEntry *result;
+ bool nowait = ((op & PGSTAT_NOWAIT) != 0);
+ bool lock_acquired = true;
+ bool found = true;
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
+ if (!IsUnderPostmaster || !pgStatDBHash)
+ return NULL;
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
+ /* Lookup or create the hash table entry for this database */
+ if (op & PGSTAT_EXCLUSIVE)
+ {
+ result = (PgStat_StatDBEntry *)
+ dshash_find_or_insert_extended(pgStatDBHash, &databaseid,
+ &found, nowait);
+ if (result == NULL)
+ lock_acquired = false;
+ else if (!found)
{
/*
- * Otherwise add the values to the existing entry.
+ * If not found, initialize the new one. This creates empty hash
+ * tables hash, too.
*/
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
+ init_dbentry(result);
+ reset_dbentry_counters(result);
}
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
}
else
{
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
+ result = (PgStat_StatDBEntry *)
+ dshash_find_extended(pgStatDBHash, &databaseid, true, nowait,
+ nowait ? &lock_acquired : NULL);
+ if (result == NULL)
+ found = false;
}
-}
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
+ /* Set return status if requested */
+ if (status)
{
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
+ if (!lock_acquired)
{
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
+ Assert(nowait);
+ *status = LOCK_FAILED;
}
+ else if (!found)
+ *status = NOT_FOUND;
else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
+ *status = FOUND;
}
+
+ return result;
}
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
+/*
+ * Lookup the hash table entry for the specified table. If no hash
+ * table entry exists, initialize it, if the create parameter is true.
+ * Else, return NULL.
*/
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
+static PgStat_StatTabEntry *
+pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create)
{
- PgStat_StatDBEntry *dbentry;
- int i;
+ PgStat_StatTabEntry *result;
+ bool found;
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
+ /* Lookup or create the hash table entry for this table */
+ if (create)
+ result = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(table, &tableoid, &found);
+ else
+ result = (PgStat_StatTabEntry *) dshash_find(table, &tableoid, false);
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
+ if (!create && !found)
+ return NULL;
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
+ /* If not found, initialize the new one. */
+ if (!found)
{
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
+ result->numscans = 0;
+ result->tuples_returned = 0;
+ result->tuples_fetched = 0;
+ result->tuples_inserted = 0;
+ result->tuples_updated = 0;
+ result->tuples_deleted = 0;
+ result->tuples_hot_updated = 0;
+ result->n_live_tuples = 0;
+ result->n_dead_tuples = 0;
+ result->changes_since_analyze = 0;
+ result->blocks_fetched = 0;
+ result->blocks_hit = 0;
+ result->vacuum_timestamp = 0;
+ result->vacuum_count = 0;
+ result->autovac_vacuum_timestamp = 0;
+ result->autovac_vacuum_count = 0;
+ result->analyze_timestamp = 0;
+ result->analyze_count = 0;
+ result->autovac_analyze_timestamp = 0;
+ result->autovac_analyze_count = 0;
}
-}
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
+ return result;
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 84fda38249..4c0ea0cc23 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -255,7 +255,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -503,7 +502,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1326,12 +1324,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1780,11 +1772,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2681,8 +2668,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3045,8 +3030,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3113,13 +3096,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3194,22 +3170,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3670,22 +3630,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3881,8 +3825,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3917,8 +3859,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4119,8 +4060,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5093,18 +5032,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5223,12 +5150,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6133,7 +6054,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6189,8 +6109,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6425,7 +6343,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 4829953ee6..5093a4a11d 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 18e3843e8b..caa00011a9 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -522,6 +522,7 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
+ LWLockRegisterTranche(LWTRANCHE_STATS, "activity stats");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 3b85e48333..c518424471 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3181,6 +3181,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3755,6 +3761,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4194,6 +4201,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4206,8 +4215,13 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle", false);
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4242,7 +4256,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4250,6 +4264,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 3bf96de256..9c694f20c9 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -32,6 +32,7 @@ volatile sig_atomic_t QueryCancelPending = false;
volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile sig_atomic_t ConfigReloadPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index cc38669a1e..a60bc58b0c 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -630,6 +631,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1240,6 +1243,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index b7d36b65dd..13be46c172 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 106;
+use Test::More tests => 105;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1f4db67f3f..43250c3885 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -82,6 +82,7 @@ extern PGDLLIMPORT volatile sig_atomic_t InterruptPending;
extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 65713abc2b..c9fbcead3f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL statistics collector facility.
*
* Copyright (c) 2001-2019, PostgreSQL Global Development Group
*
@@ -13,10 +13,11 @@
#include "datatype/timestamp.h"
#include "libpq/pqcomm.h"
-#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -40,33 +41,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -77,9 +51,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -115,13 +88,6 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_blocks_hit;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -180,236 +146,12 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
+ * PgStat_MsgBgWriter bgwriter statistics
* ----------
*/
typedef struct PgStat_MsgBgWriter
{
- PgStat_MsgHdr m_hdr;
-
PgStat_Counter m_timed_checkpoints;
PgStat_Counter m_requested_checkpoints;
PgStat_Counter m_buf_written_checkpoints;
@@ -422,38 +164,14 @@ typedef struct PgStat_MsgBgWriter
PgStat_Counter m_checkpoint_sync_time;
} PgStat_MsgBgWriter;
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
-
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -485,96 +203,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Statistic collector data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -614,16 +244,29 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_block_write_time;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ int generation; /* current generation of the below */
+ int refcnt; /* current gen reference count */
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
+ int prev_refcnt; /* prev gen reference count */
+ dshash_table_handle prev_tables; /* prev gen tables hash */
+ dshash_table_handle prev_functions; /* prev gen functions hash */
+ LWLock lock; /* Lock for the above members */
+
+ /* non-shared members */
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
} PgStat_StatDBEntry;
+#define SHARED_DBENT_SIZE offsetof(PgStat_StatDBEntry, snapshot_tables)
/* ----------
* PgStat_StatTabEntry The collector's data per table (or index)
@@ -662,7 +305,7 @@ typedef struct PgStat_StatTabEntry
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -677,7 +320,7 @@ typedef struct PgStat_StatFuncEntry
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -693,7 +336,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -779,7 +422,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_ALL,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
@@ -1214,6 +856,8 @@ extern bool pgstat_track_counts;
extern int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
@@ -1235,29 +879,26 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
-extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void pgstat_reset_all(void);
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
@@ -1429,11 +1070,13 @@ extern void pgstat_send_bgwriter(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_extended(PgStat_StatDBEntry *dbent, Oid relid);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index f9450dac90..50d0a0c9dd 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -220,6 +220,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_SXACT,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index ae5389ec96..3e02bc6f85 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.23.0
* [PATCH 4/5] Shared-memory based stats collector
@ 2019-02-21 03:44 Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2019-02-21 03:44 UTC (permalink / raw)
Previously activity statistics is shared via files on disk. Every
backend sends the numbers to the stats collector process via a socket.
It makes snapshots as a set of files on disk with a certain interval
then every backend reads them as necessary. It worked fine for
comparatively small set of statistics but the set is under the
pressure to growing up and the file size has reached the order of
megabytes. To deal with larger statistics set, this patch let backends
directly share the statistics via shared memory.
---
doc/src/sgml/monitoring.sgml | 6 +-
src/backend/postmaster/autovacuum.c | 12 +-
src/backend/postmaster/pgstat.c | 5696 ++++++++++++--------------
src/backend/postmaster/postmaster.c | 85 +-
src/backend/storage/ipc/ipci.c | 6 +
src/backend/storage/lmgr/lwlock.c | 1 +
src/backend/tcop/postgres.c | 27 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 +
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 482 +--
src/include/storage/lwlock.h | 1 +
src/include/utils/timeout.h | 1 +
14 files changed, 2682 insertions(+), 3650 deletions(-)
inline patch 1: doc/src/sgml/monitoring.sgml, src/backend/postmaster/autovacuum.c, src/backend/postmaster/pgstat.c and 11 more (+2682/-3650 lines) download inline-1.patch
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index f1df14bdea..3092c7b5a3 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -53,7 +53,6 @@ postgres 15554 0.0 0.0 57536 1184 ? Ss 18:02 0:00 postgres: back
postgres 15555 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: checkpointer
postgres 15556 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: walwriter
postgres 15557 0.0 0.0 58504 2244 ? Ss 18:02 0:00 postgres: autovacuum launcher
-postgres 15558 0.0 0.0 17512 1068 ? Ss 18:02 0:00 postgres: stats collector
postgres 15582 0.0 0.0 58772 3080 ? Ss 18:04 0:00 postgres: joe runbug 127.0.0.1 idle
postgres 15606 0.0 0.0 58772 3052 ? Ss 18:07 0:00 postgres: tgl regression [local] SELECT waiting
postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl regression [local] idle in transaction
@@ -65,9 +64,8 @@ postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl
master server process. The command arguments
shown for it are the same ones used when it was launched. The next five
processes are background worker processes automatically launched by the
- master process. (The <quote>stats collector</quote> process will not be present
- if you have set the system not to start the statistics collector; likewise
- the <quote>autovacuum launcher</quote> process can be disabled.)
+ master process. (The <quote>autovacuum launcher</quote> process will not
+ be present if you have set the system not to start it.)
Each of the remaining
processes is a server process handling one client connection. Each such
process sets its command line display in the form
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index fa875db816..c4d99d2467 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -1958,15 +1958,15 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
+ /* Start a transaction so our commands have one to play into. */
+ StartTransactionCommand();
+
/*
* may be NULL if we couldn't find an entry (only happens if we are
* forcing a vacuum for anti-wrap purposes).
*/
dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- /* Start a transaction so our commands have one to play into. */
- StartTransactionCommand();
-
/*
* Clean up any dead statistics collector entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
@@ -2749,12 +2749,10 @@ get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
if (isshared)
{
if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = pgstat_fetch_stat_tabentry_extended(shared, relid);
}
else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = pgstat_fetch_stat_tabentry_extended(dbentry, relid);
return tabentry;
}
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 6c554d9736..aba43ce1dd 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,24 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Statistics collector facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects per-table and per-function usage statistics of backends and
+ * shares them among all backends using shared memory. Every backend writes
+ * individual activity in local memory using pg_count_*() and friends
+ * interfaces during a transaction. Then pgstat_flush_stat() is called at the
+ * end of a transaction to flush out the local numbers to shared memory.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, we do that not often than
+ * PGSTAT_STAT_MIN_INTERVAL(500ms). Still it is possible that a backend
+ * cannot flush all or a part of local numbers immediately, we preserve them
+ * and try the next chance after the interval of
+ * PGSTAT_STAT_RETRY_INTERVAL(100ms), but they are not kept longer than
+ * PGSTAT_STAT_MAX_INTERVAL(1000ms).
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses stats collector creates the area then load the
+ * stored stats file if any, and the last process writes the shared stats to
+ * file then destroy the area before exit.
*
* Copyright (c) 2001-2019, PostgreSQL Global Development Group
*
@@ -19,18 +28,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "pgstat.h"
@@ -42,66 +39,38 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
-#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
-
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_STAT_MIN_INTERVAL 500 /* Minimum time between stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_STAT_RETRY_INTERVAL 100 /* Retry interval between after
+ * elapsed PGSTAT_MIN_INTERVAL*/
+#define PGSTAT_STAT_MAX_INTERVAL 1000 /* Maximum time between stats data
+ * updates; in milliseconds. */
/* ----------
* The initial size hints for the hash tables used in the collector.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -117,6 +86,19 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+/*
+ * Operation mode of pgstat_get_db_entry.
+ */
+#define PGSTAT_FETCH_SHARED 0
+#define PGSTAT_FETCH_EXCLUSIVE 1
+#define PGSTAT_FETCH_NOWAIT 2
+
+typedef enum PgStat_TableLookupState
+{
+ PGSTAT_ENTRY_NOT_FOUND,
+ PGSTAT_ENTRY_FOUND,
+ PGSTAT_ENTRY_LOCK_FAILED
+} PgStat_TableLookupState;
/* ----------
* GUC parameters
@@ -132,31 +114,65 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
+#define StatsLock (&StatsShmem->StatsMainLock)
+
+/* Shared stats bootstrap information */
+typedef struct StatsShmemStruct
+{
+ LWLock StatsMainLock;
+ dsa_handle stats_dsa_handle;
+ dshash_table_handle db_hash_handle;
+ dsa_pointer global_stats;
+ dsa_pointer archiver_stats;
+ int refcount;
+ TimestampTz last_update;
+} StatsShmemStruct;
+
/*
* BgWriter global statistics counters (unused in other processes).
* Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
+ * without needing to copy things around.
*/
-PgStat_MsgBgWriter BgWriterStats;
+PgStat_MsgBgWriter BgWriterStats = {0};
-/* ----------
- * Local data
- * ----------
- */
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
+/* Variables lives for the backend lifetime */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
+static dshash_table *pgStatDBHash = NULL;
+static MemoryContext pgSharedStatsContext = NULL;
-static struct sockaddr_storage pgStatAddr;
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
+/* dshash parameter for each type of table */
+static const dshash_parameters dsh_dbparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatDBEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+static const dshash_parameters dsh_tblparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatTabEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+static const dshash_parameters dsh_funcparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatFuncEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
/*
* Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
+ * written to shared memory.
*
* NOTE: once allocated, TabStatusArray structures are never moved or deleted
* for the life of the backend. Also, we zero out the t_id fields of the
@@ -191,8 +207,8 @@ typedef struct TabStatHashEntry
static HTAB *pgStatTabHash = NULL;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Backends store per-function info that's waiting to be flushed out to shared
+ * memory in this hash table (indexed by function OID).
*/
static HTAB *pgStatFunctions = NULL;
@@ -202,6 +218,95 @@ static HTAB *pgStatFunctions = NULL;
*/
static bool have_function_stats = false;
+
+/*
+ * local hash entries have some additional data to shared data
+ */
+
+/* common header of stats entry in backend snapshot hash */
+typedef struct PgStat_snapshot_head
+{
+ Oid key;
+ bool negative;
+ char shared_part[FLEXIBLE_ARRAY_MEMBER];
+} PgStat_snapshot_head;
+
+typedef struct PgStat_StatDBEntry_snapshot
+{
+ PgStat_snapshot_head header; /* snapshot common header */
+ PgStat_StatDBEntry shared_part; /* shared entry */
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
+} PgStat_StatDBEntry_snapshot;
+
+typedef struct PgStat_StatTabEntry_snapshot
+{
+ PgStat_snapshot_head header; /* snapshot common header */
+ PgStat_StatTabEntry shared_part; /* shared entry */
+} PgStat_StatTabEntry_snapshot;
+
+typedef struct PgStat_StatFuncEntry_snapshot
+{
+ PgStat_snapshot_head header; /* snapshot common header*/
+ PgStat_StatFuncEntry shared_part; /* shared entry */
+} PgStat_StatFuncEntry_snapshot;
+
+/* context struct for snapshot_statentry */
+typedef struct pgstat_snapshot_cxt
+{
+ char *hash_name; /* name of the snapshot hash */
+ HTAB **hash; /* placeholder for the hash */
+ int hash_entsize; /* element size of hash entry */
+ dshash_table **dshash; /* placeholder for attached dshash */
+ dshash_table_handle dsh_handle; /* dsh handle to attach */
+ const dshash_parameters *dsh_params;/* dshash params */
+} pgstat_snapshot_cxt;
+
+/*
+ * Backends store various database-wide info that's waiting to be flushed out
+ * to shared memory in these variables.
+ *
+ * checksum_failures is the exception that it stores data for all databases.
+ */
+typedef struct BackendDBStats
+{
+ int n_conflict_tablespace;
+ int n_conflict_lock;
+ int n_conflict_snapshot;
+ int n_conflict_bufferpin;
+ int n_conflict_startup_deadlock;
+ int n_deadlocks;
+ size_t n_tmpfiles;
+ size_t n_tmpfilesize;
+ HTAB *checksum_failures;
+} BackendDBStats;
+
+/* Hash entry struct for checksum_failures above */
+typedef struct ChecksumFailureEnt
+{
+ Oid dboid;
+ int count;
+} ChecksumFailureEnt;
+
+static BackendDBStats BeDBStats = {0};
+
+/* checking macros for BeDBStats */
+#define HAVE_PENDING_CONFLICTS() \
+ (BeDBStats.n_conflict_tablespace > 0 || \
+ BeDBStats.n_conflict_lock > 0 || \
+ BeDBStats.n_conflict_bufferpin > 0 || \
+ BeDBStats.n_conflict_startup_deadlock > 0)
+
+#define HAVE_PENDING_DBSTATS() \
+ (HAVE_PENDING_CONFLICTS() || \
+ BeDBStats.n_deadlocks > 0 || \
+ BeDBStats.n_tmpfiles > 0 || \
+ /* no need to check n_tmpfilesize */ \
+ BeDBStats.checksum_failures != NULL)
+
+
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
* into PgStat_TableStatus counters until we know if it is going to commit
@@ -237,36 +342,41 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot. The snapshot includes auxiliary. */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
-
-/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
-
-/* Total number of backends including auxiliary */
static int localNumBackends = 0;
-/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
- */
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
+/* Variables for activity statistics snapshot. */
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatDBEntrySnapshot;
+static TimestampTz snapshot_expires_at = 0; /* local cache expiration time */
+static bool first_in_xact = true; /* is the first time in this xact? */
+
+
+/* Context struct for pgstat_flush_* functions */
+typedef struct pgstat_flush_stat_context
+{
+ int shgeneration;
+ PgStat_StatDBEntry *shdbentry;
+ dshash_table *shdb_tabhash;
+
+ int mygeneration;
+ PgStat_StatDBEntry *mydbentry;
+ dshash_table *mydb_tabhash;
+} pgstat_flush_stat_context;
/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are not collected per database or per table.
+ * shared_* are the statistics maintained by shared statistics code and
+ * snapshot_* are backend snapshots.
*/
-static List *pending_write_requests = NIL;
-
-/* Signal handler flags */
-static volatile bool need_exit = false;
-static volatile bool got_SIGHUP = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats *snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats *snapshot_globalStats;
/*
* Total time charged to functions so far in the current backend.
@@ -280,35 +390,39 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-static void pgstat_exit(SIGNAL_ARGS);
static void pgstat_beshutdown_hook(int code, Datum arg);
-static void pgstat_sighup_handler(SIGNAL_ARGS);
-
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, int op,
+ PgStat_TableLookupState *status);
+static PgStat_StatTabEntry *pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create);
+static void pgstat_write_pgStatDBHashfile(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_pgStatDBHashfile(PgStat_StatDBEntry *dbentry);
static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
+static bool pgstat_flush_stat(pgstat_flush_stat_context *cxt, bool nowait);
+static bool pgstat_flush_tabstat(pgstat_flush_stat_context *cxt, bool nowait,
+ PgStat_TableStatus *entry);
+static bool pgstat_flush_funcstats(pgstat_flush_stat_context *cxt, bool nowait);
+static bool pgstat_flush_dbstats(pgstat_flush_stat_context *cxt, bool nowait);
+static bool pgstat_update_tabentry(dshash_table *tabhash,
+ PgStat_TableStatus *stat, bool nowait);
+static void pgstat_update_dbentry(PgStat_StatDBEntry *dbentry,
+ PgStat_TableStatus *stat);
static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
+static void pgstat_remove_useless_entries(const dshash_table_handle dshhandle,
+ const dshash_parameters *dshparams,
+ HTAB *oidtab);
static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
static void pgstat_setup_memcxt(void);
+static void pgstat_flush_recovery_conflict(PgStat_StatDBEntry *dbentry);
+static void pgstat_flush_deadlock(PgStat_StatDBEntry *dbentry);
+static void pgstat_flush_checksum_failure(PgStat_StatDBEntry *dbentry);
+static void pgstat_flush_tempfile(PgStat_StatDBEntry *dbentry);
+static HTAB *create_tabstat_hash(void);
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry_extended(PgStat_StatDBEntry *dbent, Oid funcid);
+static void pgstat_snapshot_global_stats(void);
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
@@ -316,364 +430,157 @@ static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
+/* ------------------------------------------------------------
+ * Local support functions follow
+ * ------------------------------------------------------------
+ */
+static int pin_hashes(PgStat_StatDBEntry *dbentry);
+static void unpin_hashes(PgStat_StatDBEntry *dbentry, int generation);
+static dshash_table *attach_table_hash(PgStat_StatDBEntry *dbent, int gen);
+static dshash_table *attach_function_hash(PgStat_StatDBEntry *dbent, int gen);
+static void reset_dbentry_counters(PgStat_StatDBEntry *dbentry);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute space needed for stats collector's shared memory
*/
-void
-pgstat_init(void)
+Size
+StatsShmemSize(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
-
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
- {
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
- }
-
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
-
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
-
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
-
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
-
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
-
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- test_byte++; /* just make sure variable is changed */
-
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /* If we get here, we have a working socket */
- break;
- }
-
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
-
- /*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
- */
- if (!pg_set_noblock(pgStatSock))
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
- }
-
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
- }
-
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ return sizeof(StatsShmemStruct);
}
/*
- * subroutine for pgstat_reset_all
+ * StatsShmemInit - initialize during shared-memory creation
+ */
+void
+StatsShmemInit(void)
+{
+ bool found;
+
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ }
+
+ LWLockInitialize(StatsLock, LWTRANCHE_STATS);
+
+
+}
+
+/* ----------
+ * pgstat_attach_shared_stats() -
+ *
+ * Attach to shared or create stats memory.
+ * ---------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+pgstat_attach_shared_stats(void)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ MemoryContext oldcontext;
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
+ /*
+ * Don't use dsm under postmaster, when not tracking counts, or when
+ * already attached. When area is NULL, pgstat_ functions do nothing.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster || area)
+ return;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ pgstat_setup_memcxt();
+
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ if (StatsShmem->refcount > 0)
{
- int nchars;
- Oid tmp_oid;
-
- /*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
- */
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
-
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ /* Statistics exists in shared memory. Just attach to it */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatDBHash = dshash_attach(area, &dsh_dbparams,
+ StatsShmem->db_hash_handle, 0);
}
- FreeDir(dir);
+ else
+ {
+ /* Need to create shared stats */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
+
+ /* Setup shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatDBHash = dshash_create(area, &dsh_dbparams, 0);
+
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->db_hash_handle = dshash_get_hash_table_handle(pgStatDBHash);
+ StatsShmem->last_update = 0;
+ StatsShmem->refcount = 0;
+ }
+
+ /* Setup local variables */
+ pgStatDBEntrySnapshot = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
+
+ dsa_pin_mapping(area);
+
+ /* Load data if we've just created the shared area. */
+ if (StatsShmem->refcount == 0)
+ pgstat_read_statsfiles();
+
+ StatsShmem->refcount++;
+
+ MemoryContextSwitchTo(oldcontext);
+ LWLockRelease(StatsLock);
+}
+
+/* ----------
+ * pgstat_detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file we're the last process if
+ * write_stats is true.
+ * ----------
+ */
+static void
+pgstat_detach_shared_stats(bool write_stats)
+{
+ if (!area || !IsUnderPostmaster)
+ return;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ /* write out the shared stats to file if needed */
+ if (--StatsShmem->refcount < 1 && write_stats)
+ pgstat_write_statsfiles();
+
+ /*
+ * Detach the area. Automatically destroyed when the last processes
+ * detached it.
+ */
+ dsa_detach(area);
+
+ /* We're the last process. Invalidate the dsa area handle. */
+ if (StatsShmem->refcount < 1)
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+
+ area = NULL;
+ pgStatDBHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatDBEntrySnapshot = NULL;
+ LWLockRelease(StatsLock);
}
/*
@@ -685,112 +592,18 @@ pgstat_reset_remove_files(const char *directory)
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
+ /* we must have shared stats */
+ Assert (StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
-#ifdef EXEC_BACKEND
-
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
+ /* Startup must be the only user of shared stats */
+ Assert (StatsShmem->refcount == 1);
/*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
+ * We could directly remove files and recreate the shared memory area. But
+ * detach then attach for simplicity.
*/
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
-
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
+ pgstat_detach_shared_stats(false); /* Don't write */
+ pgstat_attach_shared_stats();
}
/* ------------------------------------------------------------
@@ -798,75 +611,250 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * This requires taking some locks on the shared statistics hashes and some
+ * of updates may be postponed on lock failure. Such postponed updates are
+ * retried in later call of this function and finally cleaned up by calling
+ * this function with force = true or PGSTAT_STAT_MAX_INTERVAL milliseconds
+ * has elapsed since last cleanup. On the other hand updates by regular
+ * backends happen with the interval not shorter than
+ * PGSTAT_STAT_MIN_INTERVAL when force = false.
+ *
+ * Returns the time until the next update time in milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fair to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz last_flush = 0;
+ static TimestampTz pending_since = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
- int i;
+ pgstat_flush_stat_context cxt = {0};
+ bool pending_stats = false;
+ long elapsed;
+ long secs;
+ int usecs;
/* Don't expend a clock check if nothing to do */
if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ !HAVE_PENDING_DBSTATS() && !have_function_stats)
+ return 0;
+
+ now = GetCurrentTransactionStopTimestamp();
+
+ if (!force)
+ {
+ /*
+ * Don't flush stats unless it's been at least
+ * PGSTAT_STAT_MIN_INTERVAL msec since the last flush. Returns time
+ * to wait in the case.
+ */
+ TimestampDifference(last_flush, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs /1000;
+
+ if(elapsed < PGSTAT_STAT_MIN_INTERVAL)
+ {
+ if (pending_since == 0)
+ pending_since = now;
+
+ return PGSTAT_STAT_MIN_INTERVAL - elapsed;
+ }
+
+
+ /*
+ * Don't keep pending stats for longer than PGSTAT_STAT_MAX_INTERVAL.
+ */
+ if (pending_since > 0)
+ {
+ TimestampDifference(pending_since, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs /1000;
+
+ if(elapsed > PGSTAT_STAT_MAX_INTERVAL)
+ force = true;
+ }
+ }
+
+ /* It's the time to flush */
+ last_flush = now;
+
+ /* Flush out table stats */
+ if (pgStatTabList != NULL && !pgstat_flush_stat(&cxt, !force))
+ pending_stats = true;
+
+ /* Flush out function stats */
+ if (pgStatFunctions != NULL && !pgstat_flush_funcstats(&cxt, !force))
+ pending_stats = true;
+
+ /* Flush out database-wide stats */
+ if (HAVE_PENDING_DBSTATS())
+ {
+ if (!pgstat_flush_dbstats(&cxt, !force))
+ pending_stats = true;
+ }
+
+ /* Unpin dbentry if pinned */
+ if (cxt.mydb_tabhash)
+ {
+ dshash_detach(cxt.mydb_tabhash);
+ unpin_hashes(cxt.mydbentry, cxt.mygeneration);
+ cxt.mydb_tabhash = NULL;
+ cxt.mydbentry = NULL;
+ }
+
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (StatsShmem->last_update < last_flush)
+ StatsShmem->last_update = last_flush;
+ LWLockRelease(StatsLock);
+
+ /* record how long we keep pending stats */
+ if (pending_stats)
+ {
+ if (pending_since == 0)
+ pending_since = now;
+ return PGSTAT_STAT_RETRY_INTERVAL;
+ }
+
+ pending_since = 0;
+
+ return 0;
+}
+
+/*
+ * snapshot_statentry() - Common routine for functions
+ * pgstat_fetch_stat_*entry()
+ *
+ * Returns the pointer to the entry for the key or NULL if not found.
+ *
+ * Returned entries are stable during the current transaction or until
+ * pgstat_clear_snapshot() is called.
+ *
+ * *cxt->hash points to a HTAB* variable to store the hash for local
+ * cache. The hash is created using hash_name, hash_entsize in cxt.
+ *
+ * *cxt->dshash points to a dshash_table* variable to store attached
+ * dshash. *cxt->dsh_handle specifies the dshash to be attached.
+ */
+static void *
+snapshot_statentry(pgstat_snapshot_cxt *cxt, Oid key)
+{
+ PgStat_snapshot_head *lentry = NULL;
+ size_t keysize = cxt->dsh_params->key_size;
+ size_t dsh_entrysize = cxt->dsh_params->entry_size;
+ bool found;
/*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
+ * Create new hash with arbitrary initial number entries since we don't
+ * know how this hash will grow.
*/
- now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
+ if (!*cxt->hash)
+ {
+ HASHCTL ctl;
+
+ /* Create the hash in the stats context */
+ ctl.keysize = keysize;
+ ctl.entrysize = cxt->hash_entsize;
+ ctl.hcxt = pgStatSnapshotContext;
+ *cxt->hash = hash_create(cxt->hash_name, 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ lentry = hash_search(*cxt->hash, &key, HASH_ENTER, &found);
+
+ if (!found)
+ {
+ /* not found in local cache, search shared hash */
+
+ void *sentry;
+
+ /* attach shared hash if not given, leave it alone for later use */
+ if (!*cxt->dshash)
+ {
+ MemoryContext oldcxt;
+
+ Assert (cxt->dsh_handle != DSM_HANDLE_INVALID);
+ oldcxt = MemoryContextSwitchTo(pgStatSnapshotContext);
+ *cxt->dshash =
+ dshash_attach(area, cxt->dsh_params, cxt->dsh_handle, NULL);
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ sentry = dshash_find(*cxt->dshash, &key, false);
+
+ if (sentry)
+ {
+ /* Found. Copy it */
+ memcpy(&lentry->shared_part, sentry, dsh_entrysize);
+ dshash_release_lock(*cxt->dshash, sentry);
+
+ /* then zero out the additional space */
+ if (dsh_entrysize < cxt->hash_entsize)
+ MemSet((char *)(&lentry->shared_part) + dsh_entrysize, 0,
+ cxt->hash_entsize - dsh_entrysize -
+ offsetof(PgStat_snapshot_head, shared_part));
+ }
+
+ lentry->negative = !sentry;
+ }
+
+ if (lentry->negative)
+ return NULL;
+
+ return (void *) &lentry->shared_part;
+}
+
+/*
+ * pgstat_flush_stat: Flushes table stats out to shared statistics.
+ *
+ * If nowait is true, returns false if required lock was not acquired *
+ * immediately. In the case, infos of some tables may be left alone in
+ * TabStatusArray to wait for the next chance. cxt holds some dshash related
+ * values that we want to keep during the shared stats update. Returns true
+ * if all stats info are flushed. Caller must detach dshashes stored in cxt
+ * after use.
+ *
+ * Returns true if all entries are flushed.
+ */
+static bool
+pgstat_flush_stat(pgstat_flush_stat_context *cxt, bool nowait)
+{
+ static const PgStat_TableCounts all_zeroes;
+ TabStatusArray *tsa;
+ HTAB *new_tsa_hash = NULL;
+ TabStatusArray *dest_tsa = pgStatTabList;
+ int dest_elem = 0;
+ int i;
+
+ /* nothing to do, just return */
+ if (pgStatTabHash == NULL)
+ return true;
/*
* Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
+ * entries it points to.
*/
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
+ hash_destroy(pgStatTabHash);
pgStatTabHash = NULL;
/*
* Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * have counts, and try flushing it out to shared statistics.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
{
for (i = 0; i < tsa->tsa_used; i++)
{
PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
/* Shouldn't have any pending transaction-dependent counts */
Assert(entry->trans == NULL);
@@ -879,178 +867,344 @@ pgstat_report_stat(bool force)
sizeof(PgStat_TableCounts)) == 0)
continue;
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* try to apply the tab stats */
+ if (!pgstat_flush_tabstat(cxt, nowait, entry))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ /*
+ * Failed. Leave it alone filling at the beginning in
+ * TabStatusArray.
+ */
+ TabStatHashEntry *hash_entry;
+ bool found;
+
+ if (new_tsa_hash == NULL)
+ new_tsa_hash = create_tabstat_hash();
+
+ /* Create hash entry for this entry */
+ hash_entry = hash_search(new_tsa_hash, &entry->t_id,
+ HASH_ENTER, &found);
+ Assert(!found);
+
+ /*
+ * Move insertion pointer to the next segment. There must be
+ * enough space segments since we are just leaving some of the
+ * current elements.
+ */
+ if (dest_elem >= TABSTAT_QUANTUM)
+ {
+ Assert(dest_tsa->tsa_next != NULL);
+ dest_tsa = dest_tsa->tsa_next;
+ dest_elem = 0;
+ }
+
+ /* Move the entry if needed */
+ if (tsa != dest_tsa || i != dest_elem)
+ {
+ PgStat_TableStatus *new_entry;
+ new_entry = &dest_tsa->tsa_entries[dest_elem];
+ *new_entry = *entry;
+ entry = new_entry;
+ }
+
+ hash_entry->tsa_entry = entry;
+ dest_elem++;
}
}
- /* zero out TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
}
- /*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
- */
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
+ /* zero out unused area of TableStatus */
+ dest_tsa->tsa_used = dest_elem;
+ MemSet(&dest_tsa->tsa_entries[dest_elem], 0,
+ (TABSTAT_QUANTUM - dest_elem) * sizeof(PgStat_TableStatus));
+ while (dest_tsa->tsa_next)
+ {
+ dest_tsa = dest_tsa->tsa_next;
+ MemSet(dest_tsa->tsa_entries, 0,
+ dest_tsa->tsa_used * sizeof(PgStat_TableStatus));
+ dest_tsa->tsa_used = 0;
+ }
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ /* and set the new TabStatusArray hash if any */
+ pgStatTabHash = new_tsa_hash;
+
+ /*
+ * We no longer need shared database and table entries , but still may
+ * use that for my database.
+ */
+ if (cxt->shdb_tabhash)
+ {
+ dshash_detach(cxt->shdb_tabhash);
+ unpin_hashes(cxt->shdbentry, cxt->shgeneration);
+ cxt->shdb_tabhash = NULL;
+ cxt->shdbentry = NULL;
+ }
+
+ return pgStatTabHash == NULL;
}
-/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+/* -------
+ * Subroutines for pgstat_flush_stat.
+ * -------
*/
-static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+/*
+ * pgstat_flush_tabstat: Flushes a table stats entry.
+ *
+ * If nowait is true, returns false on lock failure. Dshashes for table and
+ * function stats are kept attached in ctx. The caller must detach them after
+ * use.
+ *
+ * Returns true if the entry is flushed.
+ */
+bool
+pgstat_flush_tabstat(pgstat_flush_stat_context *cxt, bool nowait,
+ PgStat_TableStatus *entry)
{
- int n;
- int len;
+ Oid dboid = entry->t_shared ? InvalidOid : MyDatabaseId;
+ int table_mode = PGSTAT_FETCH_EXCLUSIVE;
+ bool updated = false;
+ dshash_table *tabhash;
+ PgStat_StatDBEntry *dbent;
+ int generation;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ if (nowait)
+ table_mode |= PGSTAT_FETCH_NOWAIT;
- /*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
- */
- if (OidIsValid(tsmsg->m_databaseid))
+ /* Attach the required table hash if not yet. */
+ if ((entry->t_shared ? cxt->shdb_tabhash : cxt->mydb_tabhash) == NULL)
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
- pgStatXactCommit = 0;
- pgStatXactRollback = 0;
- pgStatBlockReadTime = 0;
- pgStatBlockWriteTime = 0;
+ /* We don't have corresponding dbentry here */
+ dbent = pgstat_get_db_entry(dboid, table_mode, NULL);
+ if (!dbent)
+ return false;
+
+ /*
+ * We don't hold dshash-lock on dbentries, since the dbentries cannot
+ * be dropped meanwhile.
+ */
+ generation = pin_hashes(dbent);
+ tabhash = attach_table_hash(dbent, generation);
+
+ if (entry->t_shared)
+ {
+ cxt->shgeneration = generation;
+ cxt->shdbentry = dbent;
+ cxt->shdb_tabhash = tabhash;
+ }
+ else
+ {
+ cxt->mygeneration = generation;
+ cxt->mydbentry = dbent;
+ cxt->mydb_tabhash = tabhash;
+
+ /*
+ * We attach mydb tabhash once per flushing. This is the chance to
+ * update database-wide stats
+ */
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
+ dbent->n_xact_commit += pgStatXactCommit;
+ dbent->n_xact_rollback += pgStatXactRollback;
+ dbent->n_block_read_time += pgStatBlockReadTime;
+ dbent->n_block_write_time += pgStatBlockWriteTime;
+ LWLockRelease(&dbent->lock);
+ pgStatXactCommit = 0;
+ pgStatXactRollback = 0;
+ pgStatBlockReadTime = 0;
+ pgStatBlockWriteTime = 0;
+ }
+ }
+ else if (entry->t_shared)
+ {
+ dbent = cxt->shdbentry;
+ tabhash = cxt->shdb_tabhash;
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ dbent = cxt->mydbentry;
+ tabhash = cxt->mydb_tabhash;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ /*
+ * dbentry is always available here, so try flush table stats first, then
+ * database stats.
+ */
+ if (pgstat_update_tabentry(tabhash, entry, nowait))
+ {
+ pgstat_update_dbentry(dbent, entry);
+ updated = true;
+ }
+
+ return updated;
}
/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+ * pgstat_flush_funcstats: Flushes function stats.
+ *
+ * If nowait is true, returns false on lock failure and leave some of the
+ * entries alone in the local hash.
+ *
+ * Returns true if all entries are flushed.
*/
-static void
-pgstat_send_funcstats(void)
+static bool
+pgstat_flush_funcstats(pgstat_flush_stat_context *cxt, bool nowait)
{
/* we assume this inits to all zeroes: */
static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
+ dshash_table *funchash;
HASH_SEQ_STATUS fstat;
+ PgStat_BackendFunctionEntry *bestat;
+ /* nothing to do, just return */
if (pgStatFunctions == NULL)
- return;
+ return true;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ /* get dbentry into cxt if not yet. */
+ if (cxt->mydbentry == NULL)
{
- PgStat_FunctionEntry *m_ent;
+ int op = PGSTAT_FETCH_EXCLUSIVE;
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
+ if (nowait)
+ op |= PGSTAT_FETCH_NOWAIT;
+
+ cxt->mydbentry = pgstat_get_db_entry(MyDatabaseId, op, NULL);
+
+ if (cxt->mydbentry == NULL)
+ return false;
+
+ cxt->mygeneration = pin_hashes(cxt->mydbentry);
+ }
+
+ funchash = attach_function_hash(cxt->mydbentry, cxt->mygeneration);
+ if (funchash == NULL)
+ return false;
+
+ have_function_stats = false;
+
+ /*
+ * Scan through the pgStatFunctions to find functions that actually have
+ * counts, and try flushing it out to shared statistics.
+ */
+ hash_seq_init(&fstat, pgStatFunctions);
+ while ((bestat = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ {
+ bool found;
+ PgStat_StatFuncEntry *funcent = NULL;
+
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&bestat->f_counts, &all_zeroes,
sizeof(PgStat_FunctionCounts)) == 0)
continue;
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
+ funcent = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert_extended(funchash, (void *) &(bestat->f_id),
+ &found, nowait);
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
+ /*
+ * We couldn't acquire lock on the required entry. Leave the local
+ * entry alone.
+ */
+ if (!funcent)
{
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
+ have_function_stats = true;
+ continue;
}
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ /* Initialize if it's new, or add to it. */
+ if (!found)
+ {
+ funcent->functionid = bestat->f_id;
+ funcent->f_numcalls = bestat->f_counts.f_numcalls;
+ funcent->f_total_time =
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_total_time);
+ funcent->f_self_time =
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_self_time);
+ }
+ else
+ {
+ funcent->f_numcalls += bestat->f_counts.f_numcalls;
+ funcent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_total_time);
+ funcent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_self_time);
+ }
+ dshash_release_lock(funchash, funcent);
+
+ /* reset used counts */
+ MemSet(&bestat->f_counts, 0, sizeof(PgStat_FunctionCounts));
}
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
+ return !have_function_stats;
}
+/*
+ * pgstat_flush_dbstats: Flushes out miscellaneous database stats.
+ *
+ * If nowait is true, returns with false on lock failure on dbentry.
+ *
+ * Returns true if all the miscellaneous stats are flushed out.
+ */
+static bool
+pgstat_flush_dbstats(pgstat_flush_stat_context *cxt, bool nowait)
+{
+ /* get dbentry if not yet. */
+ if (cxt->mydbentry == NULL)
+ {
+ int op = PGSTAT_FETCH_EXCLUSIVE;
+ if (nowait)
+ op |= PGSTAT_FETCH_NOWAIT;
+
+ cxt->mydbentry = pgstat_get_db_entry(MyDatabaseId, op, NULL);
+
+ /* Lock failure, return. */
+ if (cxt->mydbentry == NULL)
+ return false;
+
+ cxt->mygeneration = pin_hashes(cxt->mydbentry);
+ }
+
+ LWLockAcquire(&cxt->mydbentry->lock, LW_EXCLUSIVE);
+ if (HAVE_PENDING_CONFLICTS())
+ pgstat_flush_recovery_conflict(cxt->mydbentry);
+ if (BeDBStats.n_deadlocks != 0)
+ pgstat_flush_deadlock(cxt->mydbentry);
+ if (BeDBStats.n_tmpfiles != 0)
+ pgstat_flush_tempfile(cxt->mydbentry);
+ if (BeDBStats.checksum_failures != NULL)
+ pgstat_flush_checksum_failure(cxt->mydbentry);
+ LWLockRelease(&cxt->mydbentry->lock);
+
+ return true;
+}
/* ----------
* pgstat_vacuum_stat() -
*
- * Will tell the collector about objects he can get rid of.
+ * Remove objects he can get rid of.
* ----------
*/
void
pgstat_vacuum_stat(void)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
+ HTAB *oidtab;
+ dshash_seq_status dshstat;
PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* we don't collect statistics under standalone mode */
+ if (!IsUnderPostmaster)
return;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
/*
* Read pg_database and make a list of OIDs of all existing databases
*/
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ oidtab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
/*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
+ * Search the database hash table for dead databases and drop them
+ * from the hash.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+
+ dshash_seq_init(&dshstat, pgStatDBHash, false, true);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&dshstat)) != NULL)
{
Oid dbid = dbentry->databaseid;
@@ -1058,137 +1212,43 @@ pgstat_vacuum_stat(void)
/* the DB entry for shared tables (with InvalidOid) is never dropped */
if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
+ hash_search(oidtab, (void *) &dbid, HASH_FIND, NULL) == NULL)
pgstat_drop_database(dbid);
}
/* Clean up */
- hash_destroy(htab);
+ hash_destroy(oidtab);
/*
* Lookup our own database entry; if not found, nothing more to do.
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ if (!dbentry)
return;
/*
* Similarly to above, make a list of all known relations in this DB.
*/
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
+ oidtab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
/*
* Check for all tables listed in stats hashtable if they still exist.
+ * Stats cache is useless here so directly search the shared hash.
*/
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
+ pgstat_remove_useless_entries(dbentry->tables, &dsh_tblparams, oidtab);
/*
- * Send the rest
+ * Repeat the above but we needn't bother in the common case where no
+ * function stats are being collected.
*/
- if (msg.m_nentries > 0)
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
+ oidtab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
+ pgstat_remove_useless_entries(dbentry->functions, &dsh_funcparams,
+ oidtab);
}
+ dshash_release_lock(pgStatDBHash, dbentry);
}
@@ -1242,66 +1302,96 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
return htab;
}
+/*
+ * pgstat_remove_useless_entries - Remove useless entries from per
+ * table/function dshashes.
+ *
+ * Scan the dshash specified by dshhandle removing entries that are not in
+ * oidtab. oidtab is destroyed before returning.
+ */
+void
+pgstat_remove_useless_entries(const dshash_table_handle dshhandle,
+ const dshash_parameters *dshparams,
+ HTAB *oidtab)
+{
+ dshash_table *dshtable;
+ dshash_seq_status dshstat;
+ void *ent;
+
+ dshtable = dshash_attach(area, dshparams, dshhandle, 0);
+ dshash_seq_init(&dshstat, dshtable, false, true);
+
+ /* The first member of the entries must be Oid */
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hash_search(oidtab, ent, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* Not there, so purge this entry */
+ dshash_delete_entry(dshtable, ent);
+ }
+ dshash_detach(dshtable);
+ hash_destroy(oidtab);
+}
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
+ * Remove entry for the database that we just dropped.
+ *
+ * If some stats are flushed after this, this entry will re-created but we
+ * will still clean the dead DB eventually via future invocations of
+ * pgstat_vacuum_stat().
* ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
+ Assert (OidIsValid(databaseid));
+
+ if (!IsUnderPostmaster || !pgStatDBHash)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Lookup the database in the hashtable with exclusive lock.
+ */
+ dbentry = pgstat_get_db_entry(databaseid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ /*
+ * If found, remove it (along with the db statfile).
+ */
+ if (dbentry)
+ {
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ Assert(dbentry->refcnt == 0);
+
+ /* One one must live on this database. It's safe to drop all. */
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(tbl);
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(tbl);
+ }
+ LWLockRelease(&dbentry->lock);
+
+ dshash_delete_entry(pgStatDBHash, (void *)dbentry);
+ }
}
-
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
-
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-}
-#endif /* NOT_USED */
-
-
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1310,20 +1400,32 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!pgStatDBHash)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Lookup the database in the hashtable. Nothing to do if not there.
+ */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, &status);
+
+ if (!dbentry)
+ return;
+
+ /* This database is active, safe to release the lock immediately. */
+ dshash_release_lock(pgStatDBHash, dbentry);
+
+ /* Reset database-level stats. */
+ reset_dbentry_counters(dbentry);
+
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1332,29 +1434,35 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = GetCurrentTimestamp();
+ }
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ /* Reset the global background writer statistics for the cluster. */
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
+ LWLockRelease(StatsLock);
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1363,17 +1471,42 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
+ int generation;
- if (pgStatSock == PGINVALID_SOCKET)
+ dbentry = pgstat_get_db_entry(MyDatabaseId, PGSTAT_FETCH_EXCLUSIVE, NULL);
+
+ if (!dbentry)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* This database is active, safe to release the lock immediately. */
+ generation = pin_hashes(dbentry);
- pgstat_send(&msg, sizeof(msg));
+ /* Set the reset timestamp for the whole database */
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&dbentry->lock);
+
+ /* Remove object if it exists, ignore if not */
+ if (type == RESET_TABLE)
+ {
+ dshash_table *t = attach_table_hash(dbentry, generation);
+ dshash_delete_key(t, (void *) &objoid);
+ dshash_detach(t);
+ }
+
+ if (type == RESET_FUNCTION)
+ {
+ dshash_table *t = attach_function_hash(dbentry, generation);
+ if (t)
+ {
+ dshash_delete_key(t, (void *) &objoid);
+ dshash_detach(t);
+ }
+ }
+ unpin_hashes(dbentry, generation);
}
/* ----------
@@ -1387,48 +1520,81 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ /*
+ * Store the last autovacuum time in the database's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ dshash_release_lock(pgStatDBHash, dbentry);
- pgstat_send(&msg, sizeof(msg));
+ ts = GetCurrentTimestamp();
+
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->last_autovac_time = ts;
+ LWLockRelease(&dbentry->lock);
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
+ int generation;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hash table entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ generation = pin_hashes(dbentry);
+ table = attach_table_hash(dbentry, generation);
+
+ tabentry = pgstat_get_tab_entry(table, tableoid, true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = GetCurrentTimestamp();
+ tabentry->vacuum_count++;
+ }
+ dshash_release_lock(table, tabentry);
+
+ dshash_detach(table);
+ unpin_hashes(dbentry, generation);
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1439,9 +1605,14 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table *table;
+ int generation;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1470,76 +1641,156 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hashtable entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, PGSTAT_FETCH_EXCLUSIVE, NULL);
+ generation = pin_hashes(dbentry);
+ table = attach_table_hash(dbentry, generation);
+ tabentry = pgstat_get_tab_entry(table, RelationGetRelid(rel), true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+ dshash_release_lock(table, tabentry);
+
+ dshash_detach(table);
+ unpin_hashes(dbentry, generation);
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ BeDBStats.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ BeDBStats.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ BeDBStats.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ BeDBStats.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ BeDBStats.n_conflict_startup_deadlock++;
+ break;
+ }
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ /* We had a chance to flush immediately */
+ pgstat_flush_recovery_conflict(dbentry);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
+}
+
+/*
+ * flush recovery conflict stats
+ */
+static void
+pgstat_flush_recovery_conflict(PgStat_StatDBEntry *dbentry)
+{
+ dbentry->n_conflict_tablespace += BeDBStats.n_conflict_tablespace;
+ dbentry->n_conflict_lock += BeDBStats.n_conflict_lock;
+ dbentry->n_conflict_snapshot += BeDBStats.n_conflict_snapshot;
+ dbentry->n_conflict_bufferpin += BeDBStats.n_conflict_bufferpin;
+ dbentry->n_conflict_startup_deadlock += BeDBStats.n_conflict_startup_deadlock;
+
+ BeDBStats.n_conflict_tablespace = 0;
+ BeDBStats.n_conflict_lock = 0;
+ BeDBStats.n_conflict_snapshot = 0;
+ BeDBStats.n_conflict_bufferpin = 0;
+ BeDBStats.n_conflict_startup_deadlock = 0;
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ BeDBStats.n_deadlocks++;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ /* We had a chance to flush immediately */
+ pgstat_flush_deadlock(dbentry);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
}
-
-
-/* --------
- * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
- *
- * Tell the collector about one or more checksum failures.
- * --------
+/*
+ * flush dead lock stats
*/
-void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
+static void
+pgstat_flush_deadlock(PgStat_StatDBEntry *dbentry)
{
- PgStat_MsgChecksumFailure msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- pgstat_send(&msg, sizeof(msg));
+ dbentry->n_deadlocks += BeDBStats.n_deadlocks;
+ BeDBStats.n_deadlocks = 0;
}
/* --------
@@ -1557,60 +1808,153 @@ pgstat_report_checksum_failure(void)
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
-}
+ if (filesize > 0) /* Is there a case where filesize is really 0? */
+ {
+ BeDBStats.n_tmpfilesize += filesize; /* needs check overflow */
+ BeDBStats.n_tmpfiles++;
+ }
-
-/* ----------
- * pgstat_ping() -
- *
- * Send some junk data to the collector to increase traffic.
- * ----------
- */
-void
-pgstat_ping(void)
-{
- PgStat_MsgDummy msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
+ if (BeDBStats.n_tmpfiles == 0)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ return;
+
+ /* We had a chance to flush immediately */
+ pgstat_flush_tempfile(dbentry);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
}
-/* ----------
- * pgstat_send_inquiry() -
- *
- * Notify collector that we need fresh data.
- * ----------
+/*
+ * flush temporary file stats
*/
static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
+pgstat_flush_tempfile(PgStat_StatDBEntry *dbentry)
{
- PgStat_MsgInquiry msg;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
+ dbentry->n_temp_bytes += BeDBStats.n_tmpfilesize;
+ dbentry->n_temp_files += BeDBStats.n_tmpfiles;
+ BeDBStats.n_tmpfilesize = 0;
+ BeDBStats.n_tmpfiles = 0;
}
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
+ *
+ * Tell the collector about one or more checksum failures.
+ * --------
+ */
+void
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_TableLookupState status;
+ ChecksumFailureEnt *failent = NULL;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ if (BeDBStats.checksum_failures != NULL)
+ {
+ failent = hash_search(BeDBStats.checksum_failures, &dboid,
+ HASH_FIND, NULL);
+ if (failent)
+ failurecount += failent->count;
+ }
+
+ if (failurecount == 0)
+ return;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId,
+ PGSTAT_FETCH_EXCLUSIVE | PGSTAT_FETCH_NOWAIT,
+ &status);
+
+ if (status == PGSTAT_ENTRY_LOCK_FAILED)
+ {
+ if (!failent)
+ {
+ if (!BeDBStats.checksum_failures)
+ {
+ HASHCTL ctl;
+
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(ChecksumFailureEnt);
+ BeDBStats.checksum_failures =
+ hash_create("pgstat checksum failure count hash",
+ 32, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ failent = hash_search(BeDBStats.checksum_failures,
+ &dboid, HASH_ENTER, NULL);
+ }
+
+ failent->count = failurecount;
+ return;
+ }
+
+ /* We have a chance to flush immediately */
+ dbentry->n_checksum_failures += failurecount;
+ BeDBStats.checksum_failures = NULL;
+
+ dshash_release_lock(pgStatDBHash, dbentry);
+}
+
+/*
+ * flush checkpoint failure count for all databases
+ */
+static void
+pgstat_flush_checksum_failure(PgStat_StatDBEntry *dbentry)
+{
+ HASH_SEQ_STATUS stat;
+ ChecksumFailureEnt *ent;
+ bool release_dbent;
+
+ if (BeDBStats.checksum_failures == NULL)
+ return;
+
+ hash_seq_init(&stat, BeDBStats.checksum_failures);
+ while ((ent = (ChecksumFailureEnt *) hash_seq_search(&stat)) != NULL)
+ {
+ release_dbent = false;
+
+ if (dbentry->databaseid != ent->dboid)
+ {
+ dbentry = pgstat_get_db_entry(ent->dboid,
+ PGSTAT_FETCH_EXCLUSIVE, NULL);
+ if (!dbentry)
+ continue;
+
+ release_dbent = true;
+ }
+
+ dbentry->n_checksum_failures += ent->count;
+
+ if (release_dbent)
+ dshash_release_lock(pgStatDBHash, dbentry);
+ }
+
+ hash_destroy(BeDBStats.checksum_failures);
+ BeDBStats.checksum_failures = NULL;
+}
/*
* Initialize function call usage data.
@@ -1762,7 +2106,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1781,6 +2126,24 @@ pgstat_initstats(Relation rel)
rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
+/*
+ * create_tabstat_hash - create local hash as transactional storage
+ */
+static HTAB *
+create_tabstat_hash(void)
+{
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(TabStatHashEntry);
+
+ return hash_create("pgstat TabStatusArray lookup hash table",
+ TABSTAT_QUANTUM,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+}
+
/*
* get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
*/
@@ -1796,18 +2159,7 @@ get_tabstat_entry(Oid rel_id, bool isshared)
* Create hash table if we don't have it already.
*/
if (pgStatTabHash == NULL)
- {
- HASHCTL ctl;
-
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
- }
+ pgStatTabHash = create_tabstat_hash();
/*
* Find an entry or create a new one.
@@ -2161,8 +2513,8 @@ AtEOXact_PgStat(bool isCommit)
}
pgStatXactStack = NULL;
- /* Make sure any stats snapshot is thrown away */
- pgstat_clear_snapshot();
+ /* mark as the next reference is the first in a transaction */
+ first_in_xact = true;
}
/* ----------
@@ -2343,8 +2695,8 @@ PostPrepare_PgStat(void)
}
pgStatXactStack = NULL;
- /* Make sure any stats snapshot is thrown away */
- pgstat_clear_snapshot();
+ /* mark as the next reference is the first in a transaction */
+ first_in_xact = true;
}
/*
@@ -2416,30 +2768,37 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
- * ----------
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* context for snapshot_statentry */
+ static pgstat_snapshot_cxt cxt =
+ {
+ .hash_name = "local database stats hash",
+ .hash = NULL,
+ .hash_entsize = sizeof(PgStat_StatDBEntry_snapshot),
+ .dshash = NULL,
+ .dsh_handle = DSM_HANDLE_INVALID,
+ .dsh_params = &dsh_dbparams
+ };
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ cxt.dshash = &pgStatDBHash;
+ cxt.hash = &pgStatDBEntrySnapshot;
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *)
+ snapshot_statentry(&cxt, dbid);
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
@@ -2452,51 +2811,66 @@ pgstat_fetch_stat_dbentry(Oid dbid)
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* Lookup our database, then look in its table hash table. */
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
+ if (dbentry == NULL)
+ return NULL;
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_extended(dbentry, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ dbentry = pgstat_fetch_stat_dbentry(InvalidOid);
+ if (dbentry == NULL)
+ return NULL;
+
+ tabentry = pgstat_fetch_stat_tabentry_extended(dbentry, relid);
+ if (tabentry != NULL)
+ return tabentry;
return NULL;
}
+/* ----------
+ * pgstat_fetch_stat_tabentry_extended() -
+ *
+ * Find table stats entry on backends. The returned entries are cached until
+ * transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_extended(PgStat_StatDBEntry *dbent, Oid reloid)
+{
+ /* context for snapshot_statentry */
+ static pgstat_snapshot_cxt cxt =
+ {
+ .hash_name = "table stats snapshot hash",
+ .hash = NULL,
+ .hash_entsize = sizeof(PgStat_StatTabEntry_snapshot),
+ .dshash = NULL,
+ .dsh_handle = DSM_HANDLE_INVALID,
+ .dsh_params = &dsh_tblparams
+ };
+ PgStat_StatDBEntry_snapshot *local_dbent;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* dbent given to this function is alias of PgStat_StatDBEntry_snapshot */
+ local_dbent = (PgStat_StatDBEntry_snapshot *)dbent;
+ cxt.hash = &local_dbent->snapshot_tables;
+ cxt.dshash = &local_dbent->dshash_tables;
+ cxt.dsh_handle = dbent->tables;
+
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(&cxt, reloid);
+}
+
/* ----------
* pgstat_fetch_stat_funcentry() -
@@ -2511,21 +2885,127 @@ pgstat_fetch_stat_funcentry(Oid func_id)
PgStat_StatDBEntry *dbentry;
PgStat_StatFuncEntry *funcentry = NULL;
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
+ /* Lookup our database, then find the requested function */
dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
+ if (dbentry == NULL)
+ return NULL;
+
+ funcentry = pgstat_fetch_stat_funcentry_extended(dbentry, func_id);
return funcentry;
}
+/* ----------
+ * pgstat_fetch_stat_funcentry_extended() -
+ *
+ * Find function stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+static PgStat_StatFuncEntry *
+pgstat_fetch_stat_funcentry_extended(PgStat_StatDBEntry *dbent, Oid funcid)
+{
+ /* context for snapshot_statentry */
+ static pgstat_snapshot_cxt cxt =
+ {
+ .hash_name = "function stats snapshot hash",
+ .hash = NULL,
+ .hash_entsize = sizeof(PgStat_StatFuncEntry_snapshot),
+ .dshash = NULL,
+ .dsh_handle = DSM_HANDLE_INVALID,
+ .dsh_params = &dsh_funcparams
+ };
+ PgStat_StatDBEntry_snapshot *local_dbent;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ if (dbent->functions == DSM_HANDLE_INVALID)
+ return NULL;
+
+ /* dbent given to this function is alias of PgStat_StatDBEntry_snapshot */
+ local_dbent = (PgStat_StatDBEntry_snapshot *)dbent;
+ cxt.hash = &local_dbent->snapshot_functions;
+ cxt.dshash = &local_dbent->dshash_functions;
+ cxt.dsh_handle = dbent->functions;
+
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(&cxt, funcid);
+}
+
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+ TimestampTz update_time = 0;
+
+
+ pgstat_attach_shared_stats();
+ /* The snapshot lives within CacheMemoryContext */
+ if (pgStatSnapshotContext == NULL)
+ {
+ pgStatSnapshotContext =
+ AllocSetContextCreate(CacheMemoryContext,
+ "Stats snapshot context",
+ ALLOCSET_DEFAULT_SIZES);
+ }
+
+ /*
+ * Shared stats are updated frequently especially when many backends are
+ * running, but we don't want to reconstruct snapshot so frequently for
+ * performance reasons. Keep them at least for the same duration with
+ * minimal stats update interval of a backend. As the result snapshots may
+ * live for multiple transactions.
+ */
+ if (first_in_xact && IsTransactionState())
+ {
+ first_in_xact = false;
+ LWLockAcquire(StatsLock, LW_SHARED);
+ update_time = StatsShmem->last_update;
+ LWLockRelease(StatsLock);
+
+ if (snapshot_expires_at < update_time)
+ {
+ /* No problem to expire involving backend status */
+ pgstat_clear_snapshot();
+
+ snapshot_expires_at =
+ update_time + PGSTAT_STAT_MIN_INTERVAL * USECS_PER_SEC / 1000;
+ }
+ }
+
+ /* Nothing to do if already done */
+ if (snapshot_globalStats)
+ return;
+
+ Assert(snapshot_archiverStats == NULL);
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ /* global stats can be just copied */
+ LWLockAcquire(StatsLock, LW_SHARED);
+ snapshot_globalStats = palloc(sizeof(PgStat_GlobalStats));
+ memcpy(snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ snapshot_archiverStats = palloc(sizeof(PgStat_ArchiverStats));
+ memcpy(snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ LWLockRelease(StatsLock);
+
+ /* set the timestamp of this snapshot */
+ snapshot_globalStats->stats_timestamp = update_time;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
+}
/* ----------
* pgstat_fetch_stat_beentry() -
@@ -2597,9 +3077,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return snapshot_archiverStats;
}
@@ -2614,9 +3095,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return snapshot_globalStats;
}
@@ -2805,8 +3287,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutodwn */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -2977,6 +3459,8 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ pgstat_attach_shared_stats();
}
/*
@@ -3012,6 +3496,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
pgstat_increment_changecount_after(beentry);
+
+ pgstat_detach_shared_stats(true);
}
@@ -3267,7 +3753,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3534,9 +4021,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_ALL:
event_name = "RecoveryWalAll";
break;
@@ -4193,75 +4677,39 @@ pgstat_get_backend_desc(BackendType backendType)
* ------------------------------------------------------------
*/
-
-/* ----------
- * pgstat_setheader() -
- *
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
/* ----------
* pgstat_send_archiver() -
*
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Report archiver statistics
* ----------
*/
void
pgstat_send_archiver(const char *xlog, bool failed)
{
- PgStat_MsgArchiver msg;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (failed)
+ {
+ /* Failed archival attempt */
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = GetCurrentTimestamp();
+ }
+ else
+ {
+ /* Successful archival operation */
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = GetCurrentTimestamp();
+ }
+ LWLockRelease(StatsLock);
}
/* ----------
* pgstat_send_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
@@ -4270,6 +4718,8 @@ pgstat_send_bgwriter(void)
/* We assume this initializes to zeroes */
static const PgStat_MsgBgWriter all_zeroes;
+ PgStat_MsgBgWriter *s = &BgWriterStats;
+
/*
* This function can be called even if nothing at all has happened. In
* this case, avoid sending a completely empty message to the stats
@@ -4278,11 +4728,18 @@ pgstat_send_bgwriter(void)
if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += s->m_timed_checkpoints;
+ shared_globalStats->requested_checkpoints += s->m_requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += s->m_checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += s->m_checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += s->m_buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += s->m_buf_written_clean;
+ shared_globalStats->maxwritten_clean += s->m_maxwritten_clean;
+ shared_globalStats->buf_written_backend += s->m_buf_written_backend;
+ shared_globalStats->buf_fsync_backend += s->m_buf_fsync_backend;
+ shared_globalStats->buf_alloc += s->m_buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4291,302 +4748,164 @@ pgstat_send_bgwriter(void)
}
-/* ----------
- * PgstatCollectorMain() -
+/*
+ * Pin and Unpin dbentry.
*
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
+ * To keep less memory usage, and for speed, counters are by recreation of
+ * dshash instead of removing entries one-by-one keeping whole-dshash lock. On
+ * the other hand dshash cannot be destroyed until all referrers have gone. As
+ * the result, other backend may be kept waiting the counter reset for not a
+ * short time. We isolate the hashes under destruction as another generation,
+ * which means no longer used but cannot be removed yet.
+
+ * When we start accessing hashes on a dbentry, call pin_hashes() and acquire
+ * the current "generation". Unlock removes the older generation's hashes when
+ * all refers have gone.
*/
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+static int
+pin_hashes(PgStat_StatDBEntry *dbentry)
{
- int len;
- PgStat_Msg msg;
- int wr;
+ int generation;
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, pgstat_sighup_handler);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, pgstat_exit);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->refcnt++;
+ generation = dbentry->generation;
+ LWLockRelease(&dbentry->lock);
- /*
- * Identify myself via ps
- */
- init_ps_display("stats collector", "", "", "");
+ dshash_release_lock(pgStatDBHash, dbentry);
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do recognize got_SIGHUP inside
- * the inner loop, which means that such interrupts will get serviced but
- * the latch won't get cleared until next time there is a break in the
- * action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (need_exit)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * need_exit becomes set.
- */
- while (!need_exit)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (got_SIGHUP)
- {
- got_SIGHUP = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry((PgStat_MsgInquiry *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat((PgStat_MsgTabstat *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge((PgStat_MsgTabpurge *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb((PgStat_MsgDropdb *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter((PgStat_MsgResetcounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(
- (PgStat_MsgResetsharedcounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(
- (PgStat_MsgResetsinglecounter *) &msg,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac((PgStat_MsgAutovacStart *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum((PgStat_MsgVacuum *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze((PgStat_MsgAnalyze *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver((PgStat_MsgArchiver *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter((PgStat_MsgBgWriter *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat((PgStat_MsgFuncstat *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge((PgStat_MsgFuncpurge *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict((PgStat_MsgRecoveryConflict *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock((PgStat_MsgDeadlock *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile((PgStat_MsgTempFile *) &msg, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure((PgStat_MsgChecksumFailure *) &msg, len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- exit(0);
-}
-
-
-/* SIGQUIT signal handler for collector process */
-static void
-pgstat_exit(SIGNAL_ARGS)
-{
- int save_errno = errno;
-
- need_exit = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
-}
-
-/* SIGHUP handler for collector process */
-static void
-pgstat_sighup_handler(SIGNAL_ARGS)
-{
- int save_errno = errno;
-
- got_SIGHUP = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
+ return generation;
}
/*
- * Subroutine to clear stats in a database entry
+ * Unpin hashes in dbentry. If given generation is isolated, destroy it after
+ * all referrers has gone. Otherwise just decrease reference count then return.
+ */
+static void
+unpin_hashes(PgStat_StatDBEntry *dbentry, int generation)
+{
+ dshash_table *tables;
+ dshash_table *funcs = NULL;
+
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+
+ /* using current generation, just decrease refcount */
+ if (dbentry->generation == generation)
+ {
+ dbentry->refcnt--;
+ LWLockRelease(&dbentry->lock);
+ return;
+ }
+
+ /*
+ * It is isolated, waiting for all referrers to end.
+ */
+ Assert(dbentry->generation == generation + 1);
+
+ if (--dbentry->prev_refcnt > 0)
+ {
+ LWLockRelease(&dbentry->lock);
+ return;
+ }
+
+ /* no referrer remains, remove the hashes */
+ tables = dshash_attach(area, &dsh_tblparams, dbentry->prev_tables, 0);
+ if (dbentry->prev_functions != DSM_HANDLE_INVALID)
+ funcs = dshash_attach(area, &dsh_funcparams,
+ dbentry->prev_functions, 0);
+
+ dbentry->prev_tables = DSM_HANDLE_INVALID;
+ dbentry->prev_functions = DSM_HANDLE_INVALID;
+
+ /* release the entry immediately */
+ LWLockRelease(&dbentry->lock);
+
+ dshash_destroy(tables);
+ if (funcs)
+ dshash_destroy(funcs);
+
+ return;
+}
+
+/*
+ * attach and return the specified generation of table hash
+ * Returns NULL on lock failure.
+ */
+static dshash_table *
+attach_table_hash(PgStat_StatDBEntry *dbent, int gen)
+{
+ dshash_table *ret;
+
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
+
+ if (dbent->generation == gen)
+ ret = dshash_attach(area, &dsh_tblparams, dbent->tables, 0);
+ else
+ {
+ Assert (dbent->generation == gen + 1);
+ Assert (dbent->prev_tables != DSM_HANDLE_INVALID);
+ ret = dshash_attach(area, &dsh_tblparams, dbent->prev_tables, 0);
+ }
+ LWLockRelease(&dbent->lock);
+
+ return ret;
+}
+
+/* attach and return the specified generation of function hash */
+static dshash_table *
+attach_function_hash(PgStat_StatDBEntry *dbent, int gen)
+{
+ dshash_table *ret = NULL;
+
+
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
+
+ if (dbent->generation == gen)
+ {
+ if (dbent->functions == DSM_HANDLE_INVALID)
+ {
+ dshash_table *funchash =
+ dshash_create(area, &dsh_funcparams, 0);
+ dbent->functions = dshash_get_hash_table_handle(funchash);
+
+ ret = funchash;
+ }
+ else
+ ret = dshash_attach(area, &dsh_funcparams, dbent->functions, 0);
+ }
+ /* don't bother creating useless hash */
+
+ LWLockRelease(&dbent->lock);
+
+ return ret;
+}
+
+static void
+init_dbentry(PgStat_StatDBEntry *dbentry)
+{
+ LWLockInitialize(&dbentry->lock, LWTRANCHE_STATS);
+ dbentry->generation = 0;
+ dbentry->refcnt = 0;
+ dbentry->prev_refcnt = 0;
+ dbentry->tables = DSM_HANDLE_INVALID;
+ dbentry->prev_tables = DSM_HANDLE_INVALID;
+ dbentry->functions = DSM_HANDLE_INVALID;
+ dbentry->prev_functions = DSM_HANDLE_INVALID;
+}
+
+/*
+ * Subroutine to reset stats in a shared database entry
*
- * Tables and functions hashes are initialized to empty.
+ * All counters are reset. Tables and functions dshashes are destroyed. If
+ * any backend is pinning this dbentry, the current dshashes are stashed out to
+ * the previous "generation" to wait for all accessors gone. If the previous
+ * generation is already occupied, the current dshashes are so fresh that they
+ * doesn't need to be cleared.
*/
static void
reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
{
- HASHCTL hash_ctl;
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
dbentry->n_xact_commit = 0;
dbentry->n_xact_rollback = 0;
@@ -4610,23 +4929,797 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
dbentry->n_block_read_time = 0;
dbentry->n_block_write_time = 0;
+ if (dbentry->refcnt == 0)
+ {
+ /*
+ * No one is referring to the current hash. Removing individual
+ * entries in dshash is very costly so just destroy it. If someone
+ * pined this entry just after, pin_hashes returns the current
+ * generation and attach waits for the following LWLock.
+ */
+ dshash_table *tbl;
+
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_destroy(tbl);
+ dbentry->tables = DSM_HANDLE_INVALID;
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ tbl = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_destroy(tbl);
+ dbentry->functions = DSM_HANDLE_INVALID;
+ }
+ }
+ else if (dbentry->prev_refcnt == 0)
+ {
+ /*
+ * Someone is still referring to the current hash and previous slot is
+ * vacant. Stash out the current hash to the previous slot.
+ */
+ dbentry->prev_refcnt = dbentry->refcnt;
+ dbentry->prev_tables = dbentry->tables;
+ dbentry->prev_functions = dbentry->functions;
+ dbentry->refcnt = 0;
+ dbentry->tables = DSM_HANDLE_INVALID;
+ dbentry->functions = DSM_HANDLE_INVALID;
+ dbentry->generation++;
+ }
+ else
+ {
+ Assert(dbentry->prev_refcnt > 0 && dbentry->refcnt > 0);
+ /*
+ * If we get here, we just have got another reset request and the old
+ * hashes are waiting to all referrers to release. It must be quite a
+ * short time so we can just ignore this request.
+ */
+ }
+
+ /* Create new table hash if not exists */
+ if (dbentry->tables == DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables = dshash_get_hash_table_handle(tbl);
+ dshash_detach(tbl);
+ }
+
+ /* Recreate now if needed. */
+ if (dbentry->functions == DSM_HANDLE_INVALID &&
+ pgstat_track_functions != TRACK_FUNC_OFF)
+ {
+ dshash_table *tbl = dshash_create(area, &dsh_funcparams, 0);
+ dbentry->functions = dshash_get_hash_table_handle(tbl);
+ dshash_detach(tbl);
+ }
+
dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
+ LWLockRelease(&dbentry->lock);
+}
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
+/*
+ * Create the filename for a DB stat file; filename is the output buffer, of
+ * length len.
+ */
+static void
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
+{
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
+}
+
+/* ----------
+ * pgstat_write_statsfiles() -
+ * Write the global statistics file, as well as DB files.
+ * ----------
+ */
+void
+pgstat_write_statsfiles(void)
+{
+ dshash_seq_status hstat;
+ PgStat_StatDBEntry *dbentry;
+ FILE *fpout;
+ int32 format_id;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+ int rc;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
+
+ elog(DEBUG2, "writing stats file \"%s\"", statfile);
+
+ /*
+ * Open the statistics temp file to write out the current values.
+ */
+ fpout = AllocateFile(tmpfile, PG_BINARY_W);
+ if (fpout == NULL)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open temporary statistics file \"%s\": %m",
+ tmpfile)));
+ return;
+ }
+
+ /*
+ * Set the timestamp of the stats file.
+ */
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
+
+ /*
+ * Write the file header --- currently just a format ID.
+ */
+ format_id = PGSTAT_FILE_FORMAT_ID;
+ rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Write global stats struct
+ */
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Write archiver stats struct
+ */
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Walk through the database table.
+ */
+ dshash_seq_init(&hstat, pgStatDBHash, false, false);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&hstat)) != NULL)
+ {
+ /*
+ * Write out the table and function stats for this DB into the
+ * appropriate per-DB stat file, if required.
+ */
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
+
+ pgstat_write_pgStatDBHashfile(dbentry);
+
+ /*
+ * Write out the DB entry. We don't write the tables or functions
+ * pointers, since they're of no use to any other process.
+ */
+ fputc('D', fpout);
+ rc = fwrite(dbentry,
+ offsetof(PgStat_StatDBEntry, generation), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+
+ /*
+ * No more output to be done. Close the temp file and replace the old
+ * pgstat.stat with it. The ferror() check replaces testing for error
+ * after each individual fputc or fwrite above.
+ */
+ fputc('E', fpout);
+
+ if (ferror(fpout))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write temporary statistics file \"%s\": %m",
+ tmpfile)));
+ FreeFile(fpout);
+ unlink(tmpfile);
+ }
+ else if (FreeFile(fpout) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close temporary statistics file \"%s\": %m",
+ tmpfile)));
+ unlink(tmpfile);
+ }
+ else if (rename(tmpfile, statfile) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
+ tmpfile, statfile)));
+ unlink(tmpfile);
+ }
+}
+
+/* ----------
+ * pgstat_write_pgStatDBHashfile() -
+ * Write the stat file for a single database.
+ * ----------
+ */
+static void
+pgstat_write_pgStatDBHashfile(PgStat_StatDBEntry *dbentry)
+{
+ dshash_seq_status tstat;
+ dshash_seq_status fstat;
+ PgStat_StatTabEntry *tabentry;
+ PgStat_StatFuncEntry *funcentry;
+ FILE *fpout;
+ int32 format_id;
+ Oid dbid = dbentry->databaseid;
+ int rc;
+ char tmpfile[MAXPGPATH];
+ char statfile[MAXPGPATH];
+ dshash_table *tbl;
+
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
+
+ elog(DEBUG2, "writing stats file \"%s\"", statfile);
+
+ /*
+ * Open the statistics temp file to write out the current values.
+ */
+ fpout = AllocateFile(tmpfile, PG_BINARY_W);
+ if (fpout == NULL)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open temporary statistics file \"%s\": %m",
+ tmpfile)));
+ return;
+ }
+
+ /*
+ * Write the file header --- currently just a format ID.
+ */
+ format_id = PGSTAT_FILE_FORMAT_ID;
+ rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Walk through the database's access stats per table.
+ */
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&tstat, tbl, false, false);
+ while ((tabentry = (PgStat_StatTabEntry *) dshash_seq_next(&tstat)) != NULL)
+ {
+ fputc('T', fpout);
+ rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ dshash_detach(tbl);
+
+ /*
+ * Walk through the database's function stats table.
+ */
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ tbl = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_seq_init(&fstat, tbl, false, false);
+ while ((funcentry = (PgStat_StatFuncEntry *) dshash_seq_next(&fstat)) != NULL)
+ {
+ fputc('F', fpout);
+ rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ dshash_detach(tbl);
+ }
+
+ /*
+ * No more output to be done. Close the temp file and replace the old
+ * pgstat.stat with it. The ferror() check replaces testing for error
+ * after each individual fputc or fwrite above.
+ */
+ fputc('E', fpout);
+
+ if (ferror(fpout))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write temporary statistics file \"%s\": %m",
+ tmpfile)));
+ FreeFile(fpout);
+ unlink(tmpfile);
+ }
+ else if (FreeFile(fpout) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close temporary statistics file \"%s\": %m",
+ tmpfile)));
+ unlink(tmpfile);
+ }
+ else if (rename(tmpfile, statfile) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
+ tmpfile, statfile)));
+ unlink(tmpfile);
+ }
+}
+
+/* ----------
+ * pgstat_read_statsfiles() -
+ *
+ * Reads in existing statistics collector files into the shared stats hash.
+ *
+ * ----------
+ */
+void
+pgstat_read_statsfiles(void)
+{
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatDBEntry dbbuf;
+ FILE *fpin;
+ int32 format_id;
+ bool found;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
+
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
+
+ /*
+ * Set the current timestamp (will be kept only in case we can't load an
+ * existing statsfile).
+ */
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
+
+ /*
+ * Try to open the stats file. If it doesn't exist, the backends simply
+ * return zero for anything and the collector simply starts from scratch
+ * with empty counters.
+ *
+ * ENOENT is a possibility if the stats collector is not running or has
+ * not yet written the stats file the first time. Any other failure
+ * condition is suspicious.
+ */
+ if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open statistics file \"%s\": %m",
+ statfile)));
+ return;
+ }
+
+ /*
+ * Verify it's of the expected format.
+ */
+ if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
+ format_id != PGSTAT_FILE_FORMAT_ID)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
+
+ /*
+ * Read global stats struct
+ */
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ goto done;
+ }
+
+ /*
+ * In the collector, disregard the timestamp we read from the permanent
+ * stats file; we should be willing to write a temp stats file immediately
+ * upon the first request from any backend. This only matters if the old
+ * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
+ * an unusual scenario.
+ */
+ shared_globalStats->stats_timestamp = 0;
+
+ /*
+ * Read archiver stats struct
+ */
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ goto done;
+ }
+
+ /*
+ * We found an existing collector stats file. Read it and put all the
+ * hashtable entries into place.
+ */
+ for (;;)
+ {
+ switch (fgetc(fpin))
+ {
+ /*
+ * 'D' A PgStat_StatDBEntry struct describing a database
+ * follows.
+ */
+ case 'D':
+ if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, generation),
+ fpin) != offsetof(PgStat_StatDBEntry, generation))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ /*
+ * Add to the DB hash
+ */
+ dbentry = (PgStat_StatDBEntry *)
+ dshash_find_or_insert(pgStatDBHash, (void *) &dbbuf.databaseid,
+ &found);
+
+ /* don't allow duplicate dbentries */
+ if (found)
+ {
+ dshash_release_lock(pgStatDBHash, dbentry);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ /* initialize the new shared entry */
+ init_dbentry(dbentry);
+
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, generation));
+
+ /* Read the data from the database-specific file. */
+ pgstat_read_pgStatDBHashfile(dbentry);
+ dshash_release_lock(pgStatDBHash, dbentry);
+ break;
+
+ case 'E':
+ goto done;
+
+ default:
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+ }
+
+done:
+ FreeFile(fpin);
+
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
+
+ return;
+}
+
+
+/* ----------
+ * pgstat_read_pgStatDBHashfile() -
+ *
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
+ * ----------
+ */
+static void
+pgstat_read_pgStatDBHashfile(PgStat_StatDBEntry *dbentry)
+{
+ PgStat_StatTabEntry *tabentry;
+ PgStat_StatTabEntry tabbuf;
+ PgStat_StatFuncEntry funcbuf;
+ PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
+ FILE *fpin;
+ int32 format_id;
+ bool found;
+ char statfile[MAXPGPATH];
+
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
+
+ /*
+ * Try to open the stats file. If it doesn't exist, the backends simply
+ * return zero for anything and the collector simply starts from scratch
+ * with empty counters.
+ *
+ * ENOENT is a possibility if the stats collector is not running or has
+ * not yet written the stats file the first time. Any other failure
+ * condition is suspicious.
+ */
+ if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
+ {
+ if (errno != ENOENT)
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open statistics file \"%s\": %m",
+ statfile)));
+ return;
+ }
+
+ /*
+ * Verify it's of the expected format.
+ */
+ if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
+ format_id != PGSTAT_FILE_FORMAT_ID)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
+
+ /*
+ * We found an existing statistics file. Read it and put all the hashtable
+ * entries into place.
+ */
+ for (;;)
+ {
+ switch (fgetc(fpin))
+ {
+ /*
+ * 'T' A PgStat_StatTabEntry follows.
+ */
+ case 'T':
+ if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
+ fpin) != sizeof(PgStat_StatTabEntry))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ if (tabhash == NULL)
+ {
+ tabhash = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables =
+ dshash_get_hash_table_handle(tabhash);
+ }
+
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(tabhash,
+ (void *) &tabbuf.tableid, &found);
+
+ /* don't allow duplicate entries */
+ if (found)
+ {
+ dshash_release_lock(tabhash, tabentry);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ memcpy(tabentry, &tabbuf, sizeof(tabbuf));
+ dshash_release_lock(tabhash, tabentry);
+ break;
+
+ /*
+ * 'F' A PgStat_StatFuncEntry follows.
+ */
+ case 'F':
+ if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
+ fpin) != sizeof(PgStat_StatFuncEntry))
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ if (funchash == NULL)
+ {
+ funchash = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->functions =
+ dshash_get_hash_table_handle(funchash);
+ }
+
+ funcentry = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert(funchash,
+ (void *) &funcbuf.functionid, &found);
+
+ if (found)
+ {
+ dshash_release_lock(funchash, funcentry);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+
+ memcpy(funcentry, &funcbuf, sizeof(funcbuf));
+ dshash_release_lock(funchash, funcentry);
+ break;
+
+ /*
+ * 'E' The EOF marker of a complete stats file.
+ */
+ case 'E':
+ goto done;
+
+ default:
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
+ }
+
+done:
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
+
+ FreeFile(fpin);
+
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
+}
+
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgSharedStatsContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgStatLocalContext)
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+ if (!pgSharedStatsContext)
+ pgSharedStatsContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Shared activity statistics",
+ ALLOCSET_SMALL_SIZES);
+}
+
+/* ----------
+ * pgstat_clear_snapshot() -
+ *
+ * Discard any data collected in the current transaction. Any subsequent
+ * request will cause new snapshots to be read.
+ *
+ * This is also invoked during transaction commit or abort to discard
+ * the no-longer-wanted snapshot.
+ * ----------
+ */
+void
+pgstat_clear_snapshot(void)
+{
+ /* Release memory, if any was allocated */
+ if (pgStatLocalContext)
+ {
+ MemoryContextDelete(pgStatLocalContext);
+
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
+ }
+
+ if (pgStatSnapshotContext)
+ {
+ MemoryContextReset(pgStatSnapshotContext);
+
+ /* mark as the resource are not allocated */
+ snapshot_globalStats = NULL;
+ snapshot_archiverStats = NULL;
+ pgStatDBEntrySnapshot = NULL;
+ }
+}
+
+static bool
+pgstat_update_tabentry(dshash_table *tabhash, PgStat_TableStatus *stat,
+ bool nowait)
+{
+ PgStat_StatTabEntry *tabentry;
+ bool found;
+
+ if (tabhash == NULL)
+ return false;
+
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert_extended(tabhash, (void *) &(stat->t_id),
+ &found, nowait);
+
+ /* failed to acquire lock */
+ if (tabentry == NULL)
+ return false;
+
+ if (!found)
+ {
+ /*
+ * If it's a new table entry, initialize counters to the values we
+ * just got.
+ */
+ tabentry->numscans = stat->t_counts.t_numscans;
+ tabentry->tuples_returned = stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched = stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted = stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated = stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted = stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated = stat->t_counts.t_tuples_hot_updated;
+ tabentry->n_live_tuples = stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples = stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze = stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched = stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit = stat->t_counts.t_blocks_hit;
+
+ tabentry->vacuum_timestamp = 0;
+ tabentry->vacuum_count = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_count = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->analyze_count = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ tabentry->autovac_analyze_count = 0;
+ }
+ else
+ {
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ tabentry->numscans += stat->t_counts.t_numscans;
+ tabentry->tuples_returned += stat->t_counts.t_tuples_returned;
+ tabentry->tuples_fetched += stat->t_counts.t_tuples_fetched;
+ tabentry->tuples_inserted += stat->t_counts.t_tuples_inserted;
+ tabentry->tuples_updated += stat->t_counts.t_tuples_updated;
+ tabentry->tuples_deleted += stat->t_counts.t_tuples_deleted;
+ tabentry->tuples_hot_updated += stat->t_counts.t_tuples_hot_updated;
+ /* If table was truncated, first reset the live/dead counters */
+ if (stat->t_counts.t_truncated)
+ {
+ tabentry->n_live_tuples = 0;
+ tabentry->n_dead_tuples = 0;
+ }
+ tabentry->n_live_tuples += stat->t_counts.t_delta_live_tuples;
+ tabentry->n_dead_tuples += stat->t_counts.t_delta_dead_tuples;
+ tabentry->changes_since_analyze += stat->t_counts.t_changed_tuples;
+ tabentry->blocks_fetched += stat->t_counts.t_blocks_fetched;
+ tabentry->blocks_hit += stat->t_counts.t_blocks_hit;
+ }
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
+
+ dshash_release_lock(tabhash, tabentry);
+
+ return true;
+}
+
+static void
+pgstat_update_dbentry(PgStat_StatDBEntry *dbentry, PgStat_TableStatus *stat)
+{
+ /*
+ * Add per-table stats to the per-database entry, too.
+ */
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->n_tuples_returned += stat->t_counts.t_tuples_returned;
+ dbentry->n_tuples_fetched += stat->t_counts.t_tuples_fetched;
+ dbentry->n_tuples_inserted += stat->t_counts.t_tuples_inserted;
+ dbentry->n_tuples_updated += stat->t_counts.t_tuples_updated;
+ dbentry->n_tuples_deleted += stat->t_counts.t_tuples_deleted;
+ dbentry->n_blocks_fetched += stat->t_counts.t_blocks_fetched;
+ dbentry->n_blocks_hit += stat->t_counts.t_blocks_hit;
+ LWLockRelease(&dbentry->lock);
}
/*
@@ -4635,47 +5728,77 @@ reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
* Else, return NULL.
*/
static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
+pgstat_get_db_entry(Oid databaseid, int op, PgStat_TableLookupState *status)
{
PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
+ bool nowait = ((op & PGSTAT_FETCH_NOWAIT) != 0);
+ bool lock_acquired = true;
+ bool found = true;
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
+ if (!IsUnderPostmaster || !pgStatDBHash)
return NULL;
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
+ /* Lookup or create the hash table entry for this database */
+ if (op & PGSTAT_FETCH_EXCLUSIVE)
+ {
+ result = (PgStat_StatDBEntry *)
+ dshash_find_or_insert_extended(pgStatDBHash, &databaseid,
+ &found, nowait);
+ if (result == NULL)
+ lock_acquired = false;
+ else if (!found)
+ {
+ /*
+ * If not found, initialize the new one. This creates empty hash
+ * tables hash, too.
+ */
+ init_dbentry(result);
+ reset_dbentry_counters(result);
+ }
+ }
+ else
+ {
+ result = (PgStat_StatDBEntry *)
+ dshash_find_extended(pgStatDBHash, &databaseid, true, nowait,
+ nowait ? &lock_acquired : NULL);
+ if (result == NULL)
+ found = false;
+ }
+
+ /* Set return status if requested */
+ if (status)
+ {
+ if (!lock_acquired)
+ {
+ Assert(nowait);
+ *status = PGSTAT_ENTRY_LOCK_FAILED;
+ }
+ else if (!found)
+ *status = PGSTAT_ENTRY_NOT_FOUND;
+ else
+ *status = PGSTAT_ENTRY_FOUND;
+ }
return result;
}
-
/*
* Lookup the hash table entry for the specified table. If no hash
* table entry exists, initialize it, if the create parameter is true.
* Else, return NULL.
*/
static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
+pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create)
{
PgStat_StatTabEntry *result;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
/* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
+ if (create)
+ result = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(table, &tableoid, &found);
+ else
+ result = (PgStat_StatTabEntry *) dshash_find(table, &tableoid, false);
if (!create && !found)
return NULL;
@@ -4708,1701 +5831,6 @@ pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
return result;
}
-
-/* ----------
- * pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
- * ----------
- */
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
-{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- FILE *fpout;
- int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
- int rc;
-
- elog(DEBUG2, "writing stats file \"%s\"", statfile);
-
- /*
- * Open the statistics temp file to write out the current values.
- */
- fpout = AllocateFile(tmpfile, PG_BINARY_W);
- if (fpout == NULL)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not open temporary statistics file \"%s\": %m",
- tmpfile)));
- return;
- }
-
- /*
- * Set the timestamp of the stats file.
- */
- globalStats.stats_timestamp = GetCurrentTimestamp();
-
- /*
- * Write the file header --- currently just a format ID.
- */
- format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Write global stats struct
- */
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Write archiver stats struct
- */
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Walk through the database table.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- /*
- * Write out the table and function stats for this DB into the
- * appropriate per-DB stat file, if required.
- */
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
-
- pgstat_write_db_statsfile(dbentry, permanent);
- }
-
- /*
- * Write out the DB entry. We don't write the tables or functions
- * pointers, since they're of no use to any other process.
- */
- fputc('D', fpout);
- rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
- */
- fputc('E', fpout);
-
- if (ferror(fpout))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write temporary statistics file \"%s\": %m",
- tmpfile)));
- FreeFile(fpout);
- unlink(tmpfile);
- }
- else if (FreeFile(fpout) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not close temporary statistics file \"%s\": %m",
- tmpfile)));
- unlink(tmpfile);
- }
- else if (rename(tmpfile, statfile) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
- tmpfile, statfile)));
- unlink(tmpfile);
- }
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
-}
-
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
-
-/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
- * ----------
- */
-static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
-{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpout;
- int32 format_id;
- Oid dbid = dbentry->databaseid;
- int rc;
- char tmpfile[MAXPGPATH];
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "writing stats file \"%s\"", statfile);
-
- /*
- * Open the statistics temp file to write out the current values.
- */
- fpout = AllocateFile(tmpfile, PG_BINARY_W);
- if (fpout == NULL)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not open temporary statistics file \"%s\": %m",
- tmpfile)));
- return;
- }
-
- /*
- * Write the file header --- currently just a format ID.
- */
- format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Walk through the database's access stats per table.
- */
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
- {
- fputc('T', fpout);
- rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * Walk through the database's function stats table.
- */
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
- {
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
- */
- fputc('E', fpout);
-
- if (ferror(fpout))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write temporary statistics file \"%s\": %m",
- tmpfile)));
- FreeFile(fpout);
- unlink(tmpfile);
- }
- else if (FreeFile(fpout) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not close temporary statistics file \"%s\": %m",
- tmpfile)));
- unlink(tmpfile);
- }
- else if (rename(tmpfile, statfile) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
- tmpfile, statfile)));
- unlink(tmpfile);
- }
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_statsfiles() -
- *
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
- *
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
- * ----------
- */
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
- FILE *fpin;
- int32 format_id;
- bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
-
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
-
- /*
- * Set the current timestamp (will be kept only in case we can't load an
- * existing statsfile).
- */
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
- *
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
- * condition is suspicious.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return dbhash;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- goto done;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
- goto done;
- }
-
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
- /*
- * Read archiver stats struct
- */
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
- goto done;
- }
-
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Add to the DB hash
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-
- return dbhash;
-}
-
-
-/* ----------
- * pgstat_read_db_statsfile() -
- *
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
- * ----------
- */
-static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
-{
- PgStat_StatTabEntry *tabentry;
- PgStat_StatTabEntry tabbuf;
- PgStat_StatFuncEntry funcbuf;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpin;
- int32 format_id;
- bool found;
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
-
- /*
- * Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
- *
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
- * condition is suspicious.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- goto done;
- }
-
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'T' A PgStat_StatTabEntry follows.
- */
- case 'T':
- if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
- fpin) != sizeof(PgStat_StatTabEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(tabentry, &tabbuf, sizeof(tabbuf));
- break;
-
- /*
- * 'F' A PgStat_StatFuncEntry follows.
- */
- case 'F':
- if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
- fpin) != sizeof(PgStat_StatFuncEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(funcentry, &funcbuf, sizeof(funcbuf));
- break;
-
- /*
- * 'E' The EOF marker of a complete stats file.
- */
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
-}
-
-
-/* ----------
- * pgstat_setup_memcxt() -
- *
- * Create pgStatLocalContext, if not already done.
- * ----------
- */
-static void
-pgstat_setup_memcxt(void)
-{
- if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
-}
-
-
-/* ----------
- * pgstat_clear_snapshot() -
- *
- * Discard any data collected in the current transaction. Any subsequent
- * request will cause new snapshots to be read.
- *
- * This is also invoked during transaction commit or abort to discard
- * the no-longer-wanted snapshot.
- * ----------
- */
-void
-pgstat_clear_snapshot(void)
-{
- /* Release memory, if any was allocated */
- if (pgStatLocalContext)
- MemoryContextDelete(pgStatLocalContext);
-
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetshared() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
-}
-
/*
* Convert a potentially unsafely truncated activity string (see
* PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 36e49b3e9e..392112dea9 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -255,7 +255,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -502,7 +501,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1302,12 +1300,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1756,11 +1748,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2612,8 +2599,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -2944,8 +2929,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3012,13 +2995,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3093,22 +3069,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3567,22 +3527,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3778,8 +3722,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3818,8 +3760,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4020,8 +3961,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -4994,18 +4933,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5118,12 +5045,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5993,7 +5914,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6046,8 +5966,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6279,7 +6197,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 5965d3620f..97bca9be24 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -150,6 +150,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -281,8 +282,13 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
/* Initialize dynamic shared memory facilities. */
if (!IsUnderPostmaster)
+ {
dsm_postmaster_startup(shim);
+ /* Stats collector uses dynamic shared memory */
+ StatsShmemInit();
+ }
+
/*
* Now give loadable modules a chance to set up their shmem allocations
*/
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index bc1aa88322..b9c33d6044 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -522,6 +522,7 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
+ LWLockRegisterTranche(LWTRANCHE_STATS, "activity stats");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index f9ce3d8f22..734631dc61 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3150,6 +3150,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3724,6 +3730,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4164,9 +4171,17 @@ PostgresMain(int argc, char *argv[],
}
else
{
- ProcessCompletedNotifies();
- pgstat_report_stat(false);
+ long stats_timeout;
+ ProcessCompletedNotifies();
+
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle", false);
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4201,7 +4216,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4209,6 +4224,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index a5950c1e8c..b24631b7b1 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -32,6 +32,7 @@ volatile sig_atomic_t QueryCancelPending = false;
volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile sig_atomic_t ConfigReloadPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 752010ed27..a635f46630 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -629,6 +630,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1239,6 +1242,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 33869fecc9..8939758c59 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index c33abee7aa..6aa9e3c121 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -82,6 +82,7 @@ extern PGDLLIMPORT volatile sig_atomic_t InterruptPending;
extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e99cd4a72e..5d15026db3 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL statistics collector facility.
*
* Copyright (c) 2001-2019, PostgreSQL Global Development Group
*
@@ -14,10 +14,11 @@
#include "datatype/timestamp.h"
#include "fmgr.h"
#include "libpq/pqcomm.h"
-#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,33 +42,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -116,13 +90,6 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_blocks_hit;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -181,236 +148,12 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
+ * PgStat_MsgBgWriter bgwriter statistics
* ----------
*/
typedef struct PgStat_MsgBgWriter
{
- PgStat_MsgHdr m_hdr;
-
PgStat_Counter m_timed_checkpoints;
PgStat_Counter m_requested_checkpoints;
PgStat_Counter m_buf_written_checkpoints;
@@ -423,38 +166,14 @@ typedef struct PgStat_MsgBgWriter
PgStat_Counter m_checkpoint_sync_time;
} PgStat_MsgBgWriter;
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
-
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to apply.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when applying to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -486,93 +205,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Statistic collector data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -581,6 +215,12 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9D
+typedef struct PgStat_DSHash
+{
+ int refcnt;
+ dshash_table_handle handle;
+} PgStat_DSHash;
+
/* ----------
* PgStat_StatDBEntry The collector's data per database
* ----------
@@ -611,17 +251,22 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_block_write_time;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ int generation; /* current generation of the below */
+ int refcnt; /* current gen reference count */
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
+ int prev_refcnt; /* prev gen reference count */
+ dshash_table_handle prev_tables; /* prev gen tables hash */
+ dshash_table_handle prev_functions; /* prev gen functions hash */
+ LWLock lock; /* Lock for the above members */
} PgStat_StatDBEntry;
-
/* ----------
* PgStat_StatTabEntry The collector's data per table (or index)
* ----------
@@ -659,7 +304,7 @@ typedef struct PgStat_StatTabEntry
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -674,7 +319,7 @@ typedef struct PgStat_StatFuncEntry
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -690,7 +335,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -776,7 +421,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_ALL,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
@@ -1163,6 +807,8 @@ extern bool pgstat_track_counts;
extern int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
@@ -1184,31 +830,29 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern void pgstat_initialize(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
-extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
-
+extern void pgstat_reset_shared_counters(const char *target);
+extern void pgstat_reset_single_counter(Oid objectid,
+ PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples);
@@ -1221,26 +865,10 @@ extern void pgstat_report_deadlock(void);
extern void pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount);
extern void pgstat_report_checksum_failure(void);
-extern void pgstat_initialize(void);
extern void pgstat_bestart(void);
-
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
-extern void pgstat_report_tempfile(size_t filesize);
-extern void pgstat_report_appname(const char *appname);
-extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
-extern const char *pgstat_get_wait_event(uint32 wait_event_info);
-extern const char *pgstat_get_wait_event_type(uint32 wait_event_info);
-extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
-extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
- int buflen);
-extern const char *pgstat_get_backend_desc(BackendType backendType);
-extern void pgstat_progress_start_command(ProgressCommandType cmdtype,
- Oid relid);
-extern void pgstat_progress_update_param(int index, int64 val);
-extern void pgstat_progress_update_multi_param(int nparam, const int *index,
- const int64 *val);
-extern void pgstat_progress_end_command(void);
+extern void pgstat_report_tempfile(size_t filesize);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
@@ -1370,18 +998,38 @@ extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
extern void pgstat_send_archiver(const char *xlog, bool failed);
extern void pgstat_send_bgwriter(void);
+
+
+extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid relid);
+
+extern void pgstat_report_appname(const char *appname);
+extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
+extern const char *pgstat_get_wait_event(uint32 wait_event_info);
+extern const char *pgstat_get_wait_event_type(uint32 wait_event_info);
+extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
+extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
+ int buflen);
+extern const char *pgstat_get_backend_desc(BackendType backendType);
+
+extern void pgstat_progress_start_command(ProgressCommandType cmdtype,
+ Oid relid);
+extern void pgstat_progress_update_param(int index, int64 val);
+extern void pgstat_progress_update_multi_param(int nparam, const int *index,
+ const int64 *val);
+extern void pgstat_progress_end_command(void);
/* ----------
* Support functions for the SQL-callable functions to
* generate the pgstat* views.
* ----------
*/
-extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
+extern void pgstat_clear_snapshot(void);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
-extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
-extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_extended(PgStat_StatDBEntry *dbent, Oid relid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
-extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
+extern int pgstat_fetch_stat_numbackends(void);
+extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 08e0dc8144..30d5fb63c5 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -220,6 +220,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_SXACT,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 9244a2a7b7..a9b625211b 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.16.3
* [PATCH v45 4/7] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 1 second and such large data
is serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 10 seconds.
Until 60 second has elapsed since the last flushing to shared stats,
lock failure postpones stats flushing to try to alleviate lock
contention that slows down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/heap/heapam_handler.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/access/transam/xlog.c | 6 +-
src/backend/catalog/index.c | 24 +-
src/backend/commands/analyze.c | 8 +-
src/backend/commands/dbcommands.c | 2 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/vacuum.c | 4 +-
src/backend/postmaster/autovacuum.c | 76 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 26 +-
src/backend/postmaster/pgarch.c | 12 +-
src/backend/postmaster/pgstat.c | 6135 +++++++----------
src/backend/postmaster/postmaster.c | 88 +-
src/backend/replication/basebackup.c | 4 +-
src/backend/replication/slot.c | 12 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 4 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/smgr.c | 4 +-
src/backend/tcop/postgres.c | 41 +-
src/backend/utils/adt/pgstatfuncs.c | 95 +-
src/backend/utils/cache/relcache.c | 5 +
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 3 -
src/backend/utils/init/postinit.c | 11 +
src/backend/utils/misc/guc.c | 14 +-
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 3 +-
src/include/pgstat.h | 727 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/guc_tables.h | 2 +-
src/include/utils/timeout.h | 1 +
35 files changed, 2791 insertions(+), 4553 deletions(-)
inline patch 1: src/backend/access/heap/heapam_handler.c, src/backend/access/heap/vacuumlazy.c, src/backend/access/transam/xlog.c and 32 more (+2791/-4553 lines) download inline-1.patch
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 10ddde4ecf..98df20ce96 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1086,8 +1086,8 @@ heapam_scan_analyze_next_tuple(TableScanDesc scan, TransactionId OldestXmin,
* our own. In this case we should count and sample the row,
* to accommodate users who load a table and analyze it in one
* transaction. (pgstat_report_analyze has to adjust the
- * numbers we send to the stats collector to make this come
- * out right.)
+ * numbers we report to the activity stats facility to make
+ * this come out right.)
*/
if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(targtuple->t_data)))
{
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f3d2265fad..ceccb7e19e 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -598,7 +598,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
new_min_multi,
false);
- /* report results to the stats collector, too */
+ /* report results to the activity stats facility, too */
pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
Max(new_live_tuples, 0),
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ede93ad7fd..5d775ba84c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2207,7 +2207,7 @@ AdvanceXLInsertBuffer(XLogRecPtr upto, bool opportunistic)
WriteRqst.Flush = 0;
XLogWrite(WriteRqst, false);
LWLockRelease(WALWriteLock);
- WalStats.m_wal_buffers_full++;
+ WalStats.wal_buffers_full++;
TRACE_POSTGRESQL_WAL_BUFFER_WRITE_DIRTY_DONE();
}
/* Re-acquire WALBufMappingLock and retry */
@@ -8592,8 +8592,8 @@ LogCheckpointEnd(bool restartpoint)
CheckpointStats.ckpt_sync_end_t);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time += write_msecs;
- BgWriterStats.m_checkpoint_sync_time += sync_msecs;
+ CheckPointerStats.checkpoint_write_time += write_msecs;
+ CheckPointerStats.checkpoint_sync_time += sync_msecs;
/*
* All of the published timing statistics are accounted for. Only
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cffbc0ac38..3f55c34909 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1854,28 +1854,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be flushed by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Copy data of pg_statistic from the old index to the new one */
CopyStatistics(oldIndexId, newIndexId);
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 7295cf0215..308b4ab034 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -644,10 +644,10 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
}
/*
- * Report ANALYZE to the stats collector, too. However, if doing
- * inherited stats we shouldn't report, because the stats collector only
- * tracks per-table stats. Reset the changes_since_analyze counter only
- * if we analyzed all columns; otherwise, there is still work for
+ * Report ANALYZE to the activity stats facility, too. However, if doing
+ * inherited stats we shouldn't report, because the activity stats facility
+ * only tracks per-table stats. Reset the changes_since_analyze counter
+ * only if we analyzed all columns; otherwise, there is still work for
* auto-analyze to do.
*/
if (!inh)
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 2b159b60eb..acea4de382 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -971,7 +971,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
DropDatabaseBuffers(db_id);
/*
- * Tell the stats collector to forget it immediately, too.
+ * Tell the active stats facility to forget it immediately, too.
*/
pgstat_drop_database(db_id);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..1464b97c7f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -336,10 +336,10 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
/*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
+ * Inform activity stats facility about our activity: basically, we
+ * truncated the matview and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
*/
pgstat_count_truncate(matviewRel);
if (!stmt->skipData)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index b97d48ee01..e80ab5edd0 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -319,8 +319,8 @@ vacuum(List *relations, VacuumParams *params,
errmsg("VACUUM option DISABLE_PAGE_SKIPPING cannot be used with FULL")));
/*
- * Send info about dead objects to the statistics collector, unless we are
- * in autovacuum --- autovacuum.c does this for itself.
+ * Send info about dead objects to the activity statistics facility, unless
+ * we are in autovacuum --- autovacuum.c does this for itself.
*/
if ((params->options & VACOPT_VACUUM) && !IsAutoVacuumWorkerProcess())
pgstat_vacuum_stat();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 47e60ca561..194a02024b 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -342,9 +342,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1684,12 +1681,12 @@ AutoVacWorkerMain(int argc, char *argv[])
char dbname[NAMEDATALEN];
/*
- * Report autovac startup to the stats collector. We deliberately do
- * this before InitPostgres, so that the last_autovac_time will get
- * updated even if the connection attempt fails. This is to prevent
- * autovac from getting "stuck" repeatedly selecting an unopenable
- * database, rather than making any progress on stuff it can connect
- * to.
+ * Report autovac startup to the activity stats facility. We
+ * deliberately do this before InitPostgres, so that the
+ * last_autovac_time will get updated even if the connection attempt
+ * fails. This is to prevent autovac from getting "stuck" repeatedly
+ * selecting an unopenable database, rather than making any progress on
+ * stuff it can connect to.
*/
pgstat_report_autovac(dbid);
@@ -1961,8 +1958,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1981,17 +1976,11 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
/*
- * Clean up any dead statistics collector entries for this DB. We always
+ * Clean up any dead activity statistics entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
* nothing worth vacuuming in the database.
*/
@@ -2034,9 +2023,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2114,8 +2100,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2198,8 +2184,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2758,29 +2744,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2984,17 +2947,10 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared = NULL;
- PgStat_StatDBEntry *dbentry = NULL;
-
- if (classForm->relisshared)
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- else
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -3024,7 +2980,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
*
* For analyze, the analysis done is that the number of tuples inserted,
* deleted and updated since the last analyze exceeds a threshold calculated
- * in the same fashion as above. Note that the collector actually stores
+ * in the same fashion as above. Note that the activity statistics stores
* the number of tuples (both live and dead) that there were as of the last
* analyze. This is asymmetric to the VACUUM case.
*
@@ -3034,8 +2990,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
*
* A table whose autovacuum_enabled option is false is
* automatically skipped (unless we have to vacuum it due to freeze_max_age).
- * Thus autovacuum can be disabled for specific tables. Also, when the stats
- * collector does not have data about a table, it will be skipped.
+ * Thus autovacuum can be disabled for specific tables. Also, when the activity
+ * statistics does not have data about a table, it will be skipped.
*
* A table whose vac_base_thresh value is < 0 takes the base value from the
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 715d5195bb..679992dc89 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -244,9 +244,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Send off activity statistics to the activity stats facility
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 54a818bf61..309a9997e1 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -360,7 +360,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ CheckPointerStats.requested_checkpoints++;
}
/*
@@ -374,7 +374,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ CheckPointerStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -495,17 +495,11 @@ CheckpointerMain(void)
/* Check for archive_timeout and switch xlog files if necessary. */
CheckArchiveTimeout();
- /*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
- */
- pgstat_send_bgwriter();
+ /* Send off activity statistics to the activity stats facility. */
+ pgstat_report_checkpointer();
/* Send WAL statistics to the stats collector. */
- pgstat_send_wal();
+ pgstat_report_wal();
/*
* If any checkpoint flags have been set, redo the loop to handle the
@@ -711,9 +705,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Report interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_report_checkpointer();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1257,8 +1251,10 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ CheckPointerStats.buf_written_backend
+ += CheckpointerShmem->num_backend_writes;
+ CheckPointerStats.buf_fsync_backend
+ += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 9a2e21bf86..504443dcc0 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -372,20 +372,20 @@ pgarch_ArchiverCopyLoop(void)
pgarch_archiveDone(xlog);
/*
- * Tell the collector about the WAL file that we successfully
- * archived
+ * Tell the activity statistics facility about the WAL file
+ * that we successfully archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
else
{
/*
- * Tell the collector about the WAL file that we failed to
- * archive
+ * Tell the activity statistics facility about the WAL file
+ * that we failed to archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 3f24a33ef1..ecf9d9adcc 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (10000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (1000ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (60000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2021, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,13 +35,9 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
-#include "executor/instrument.h"
+#include "common/hashfn.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
@@ -54,20 +45,16 @@
#include "postmaster/postmaster.h"
#include "replication/slot.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
+#include "storage/condition_variable.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
#include "utils/timestamp.h"
@@ -75,35 +62,20 @@
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 10000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 1000 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 60000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -118,7 +90,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -133,17 +104,11 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
-/*
- * BgWriter and WAL global statistics counters.
- * Stored directly in a stats message structure so they can be sent
- * without needing to copy things around. We assume these init to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-PgStat_MsgWal WalStats;
-
/*
* WAL usage counters saved from pgWALUsage at the previous call to
* pgstat_send_wal(). This is used to calculate how much WAL usage
@@ -170,73 +135,246 @@ static const char *const slru_names[] = {
#define SLRU_NUM_ELEMENTS lengthof(slru_names)
+/* struct for shared SLRU stats */
+typedef struct PgStatSharedSLRUStats
+{
+ PgStat_SLRUStats entry[SLRU_NUM_ELEMENTS];
+ LWLock lock;
+ pg_atomic_uint32 changecount;
+} PgStatSharedSLRUStats;
+
+StaticAssertDecl(sizeof(TimestampTz) == sizeof(pg_atomic_uint64),
+ "size of pg_atomic_uint64 doesn't match TimestampTz");
+
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ int refcount; /* # of processes that is attaching the
+ * shared stats memory */
+ /* Global stats structs */
+ PgStat_Archiver archiver_stats;
+ pg_atomic_uint32 archiver_changecount;
+ PgStat_BgWriter bgwriter_stats;
+ pg_atomic_uint32 bgwriter_changecount;
+ PgStat_CheckPointer checkpointer_stats;
+ pg_atomic_uint32 checkpointer_changecount;
+ PgStat_Wal wal_stats;
+ LWLock wal_stats_lock;
+ PgStatSharedSLRUStats slru_stats;
+ pg_atomic_uint32 slru_changecount;
+ pg_atomic_uint64 stats_timestamp;
+
+ /* Reset offsets, protected by StatsLock */
+ PgStat_Archiver archiver_reset_offset;
+ PgStat_BgWriter bgwriter_reset_offset;
+ PgStat_CheckPointer checkpointer_reset_offset;
+
+ /* file read/write protection */
+ bool attach_holdoff;
+ ConditionVariable holdoff_cv;
+
+ pg_atomic_uint64 gc_count; /* # of entries deleted. not
+ * protected by StatsLock */
+} StatsShmemStruct;
+
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
+
+/* CheckPointer global statistics counters */
+PgStat_CheckPointer CheckPointerStats = {0};
+
+/* WAL global statistics counters */
+PgStat_Wal WalStats = {0} ;
+
/*
- * SLRU statistics counts waiting to be sent to the collector. These are
- * stored directly in stats message format so they can be sent without needing
- * to copy things around. We assume this variable inits to zeroes. Entries
- * are one-to-one with slru_names[].
+ * XXXX: always try to flush WAL stats. We don't want to manipulate another
+ * counter during XLogInsert so we don't have an effecient short cut to know
+ * whether any counter gets incremented.
*/
-static PgStat_MsgSLRU SLRUStats[SLRU_NUM_ELEMENTS];
+static inline bool
+walstats_pending(void)
+{
+ static const PgStat_Wal all_zeroes;
+
+ return memcmp(&WalStats, &all_zeroes,
+ offsetof(PgStat_Wal, stat_reset_timestamp)) != 0;
+}
+
+/*
+ * SLRU statistics counts waiting to be written to the shared activity
+ * statistics. We assume this variable inits to zeroes. Entries are
+ * one-to-one with slru_names[].
+ * Changes of SLRU counters are reported within critical sections so we use
+ * static memory in order to avoid memory allocation.
+ */
+static PgStat_SLRUStats local_SLRUStats[SLRU_NUM_ELEMENTS];
+static bool have_slrustats = false;
/* ----------
* Local data
* ----------
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
-
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
- *
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Types to define shared statistics structure.
+ *
+ * Per-object statistics are stored in a "shared stats", corresponding struct
+ * that has a header part common among all object types in DSA-allocated
+ * memory. All shared stats are pointed from a dshash via a dsa_pointer. This
+ * structure make the shared stats immovable against dshash resizing, allows a
+ * backend point to shared stats entries via a native pointer and allows
+ * locking at stats-entry level. The per-entry locking reduces lock contention
+ * compared to partition lock of dshash. A backend accumulates stats numbers in
+ * a stats entry in the local memory space then flushes the numbers to shared
+ * stats entries at basically transaction end.
+ *
+ * Each stat entry type has a fixed member PgStat_HashEntryHeader as the first
+ * element.
+ *
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStat_Stat*Entry (dsa memory block)
+ *
+ * Shared stats entries are directly pointed from pgstat_localhash hash:
+ *
+ * pgstat_localhash pgStatEntHash
+ * -> PgStatLocalHashEntry (equivalent of PgStatHashEntry)
+ * (native pointer)-> PgStat_Stat*Entry (dsa memory block)
+ *
+ * Local stats that are waiting for being flushed to share stats are stored as:
+ *
+ * pgstat_localhash pgStatLocalHash
+ * -> PgStatLocalHashEntry (local hash entry)
+ * (native pointer)-> PgStat_Stat*Entry/TableStatus (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION, /* per-function statistics */
+ PGSTAT_TYPE_REPLSLOT /* per-replication-slot statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry body size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static const size_t pgstat_sharedentsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry), /* PGSTAT_TYPE_FUNCTION */
+ sizeof(PgStat_ReplSlot) /* PGSTAT_TYPE_REPLSLOT */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static const size_t pgstat_localentsize[] =
+{
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry),/* PGSTAT_TYPE_FUNCTION */
+ sizeof(PgStat_ReplSlot) /* PGSTAT_TYPE_REPLSLOT */
+};
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * We shoud avoid overwriting header part of a shared entry. Use these macros
+ * to know what portion of the struct to be written or read. PSTAT_SHENT_BODY
+ * returns a bit smaller address than the actual address of the next member but
+ * that doesn't matter.
*/
-static HTAB *pgStatFunctions = NULL;
+#define PGSTAT_SHENT_BODY(e) (((char *)(e)) + sizeof(PgStat_StatEntryHeader))
+#define PGSTAT_SHENT_BODY_LEN(t) \
+ (pgstat_sharedentsize[t] - sizeof(PgStat_StatEntryHeader))
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object ID, either table or function. */
+} PgStatHashKey;
+
+/* struct for shared statistics hash entry */
+typedef struct PgStatHashEntry
+{
+ PgStatHashKey key; /* hash key */
+ dsa_pointer body;/* pointer to shared stats in PgStat_StatEntryHeader */
+} PgStatHashEntry;
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashKey key; /* hash key */
+ char status; /* for simplehash use */
+ PgStat_StatEntryHeader *body; /* pointer to stats body in local heap */
+ dsa_pointer dsapointer; /* dsa pointer of body */
+} PgStatLocalHashEntry;
+
+/* parameter for the shared hash */
+static const dshash_parameters dsh_params = {
+ sizeof(PgStatHashKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* define hashtable for local hashes */
+#define SH_PREFIX pgstat_localhash
+#define SH_ELEMENT_TYPE PgStatLocalHashEntry
+#define SH_KEY_TYPE PgStatHashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ hash_bytes((unsigned char *)&key, sizeof(PgStatHashKey))
+#define SH_EQUAL(tb, a, b) (memcmp(&a, &b, sizeof(PgStatHashKey)) == 0)
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * The local cache to index shared stats entries.
+ *
+ * This is a local hash to store native pointers to shared hash
+ * entries. pgStatEntHashAge is copied from StatsShmem->gc_count at creation
+ * and garbage collection.
*/
-static bool have_function_stats = false;
+static pgstat_localhash_hash *pgStatEntHash = NULL;
+static int pgStatEntHashAge = 0; /* cache age of pgStatEntHash */
+
+/* Local stats numbers are stored here. */
+static pgstat_localhash_hash *pgStatLocalHash = NULL;
+
+/* entry type for oid hash */
+typedef struct pgstat_oident
+{
+ Oid oid;
+ char status;
+} pgstat_oident;
+
+/* Define hashtable for OID hashes. */
+StaticAssertDecl(sizeof(Oid) == 4, "oid is not compatible with uint32");
+#define SH_PREFIX pgstat_oid
+#define SH_ELEMENT_TYPE pgstat_oident
+#define SH_KEY_TYPE Oid
+#define SH_KEY oid
+#define SH_HASH_KEY(tb, key) hash_bytes_uint32(key)
+#define SH_EQUAL(tb, a, b) (a == b)
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -273,11 +411,8 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -286,23 +421,9 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Make our own memory context to make it easy to track memory usage.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-static PgStat_WalStats walStats;
-static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
-static PgStat_ReplSlotStats *replSlotStats;
-static int nReplSlotStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+MemoryContext pgStatCacheContext = NULL;
/*
* Total time charged to functions so far in the current backend.
@@ -311,40 +432,57 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/* Simple caching feature for pgstatfuncs */
+static PgStatHashKey stathashkey_zero = {0};
+static PgStatHashKey cached_dbent_key = {0};
+static PgStat_StatDBEntry cached_dbent;
+static PgStatHashKey cached_tabent_key = {0};
+static PgStat_StatTabEntry cached_tabent;
+static PgStatHashKey cached_funcent_key = {0};
+static PgStat_StatFuncEntry cached_funcent;
+
+static PgStat_Archiver cached_archiverstats;
+static bool cached_archiverstats_is_valid = false;
+static PgStat_BgWriter cached_bgwriterstats;
+static bool cached_bgwriterstats_is_valid = false;
+static PgStat_CheckPointer cached_checkpointerstats;
+static bool cached_checkpointerstats_is_valid = false;
+static PgStat_Wal cached_walstats;
+static bool cached_walstats_is_valid = false;
+static PgStat_SLRUStats cached_slrustats;
+static bool cached_slrustats_is_valid = false;
+static PgStat_ReplSlot *cached_replslotstats = NULL;
+static int n_cached_replslotstats = -1;
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static void pgstat_write_statsfile(void);
+
+static void pgstat_read_statsfile(void);
static void pgstat_read_current_status(void);
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
+static PgStat_StatEntryHeader * get_stat_entry(PgStatTypes type, Oid dbid,
+ Oid objid, bool nowait,
+ bool create, bool *found);
-static int pgstat_replslot_index(const char *name, bool create_it);
-static void pgstat_reset_replslot(int i, TimestampTz ts);
+static bool flush_tabstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_funcstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_dbstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_walstat(bool nowait);
+static bool flush_slrustat(bool nowait);
+static void delete_current_stats_entry(dshash_seq_status *hstat);
+static PgStat_StatEntryHeader * get_local_stat_entry(PgStatTypes type, Oid dbid,
+ Oid objid, bool create,
+ bool *found);
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static void pgstat_send_slru(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
+static pgstat_oid_hash *collect_oids(Oid catalogid, AttrNumber anum_oid);
static void pgstat_setup_memcxt(void);
@@ -354,491 +492,645 @@ static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
-static void pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_wal(PgStat_MsgWal *msg, int len);
-static void pgstat_recv_slru(PgStat_MsgSLRU *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
+ bool found;
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(), &found);
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ ConditionVariableInit(&StatsShmem->holdoff_cv);
+ pg_atomic_init_u32(&StatsShmem->archiver_changecount, 0);
+ pg_atomic_init_u32(&StatsShmem->bgwriter_changecount, 0);
+ pg_atomic_init_u32(&StatsShmem->checkpointer_changecount, 0);
+
+ pg_atomic_init_u64(&StatsShmem->gc_count, 0);
+
+ LWLockInitialize(&StatsShmem->wal_stats_lock, LWTRANCHE_STATS);
+ }
+}
+
+/* ----------
+ * allow_next_attacher() -
+ *
+ * Let other processes to go ahead attaching the shared stats area.
+ * ----------
+ */
+static void
+allow_next_attacher(void)
+{
+ bool triggerd = false;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (StatsShmem->attach_holdoff)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
+ StatsShmem->attach_holdoff = false;
+ triggerd = true;
}
+ LWLockRelease(StatsLock);
+
+ if (triggerd)
+ ConditionVariableBroadcast(&StatsShmem->holdoff_cv);
+}
+
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read the saved statistics file if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
/*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
+ * Don't use dsm under postmaster, or when not tracking counts.
*/
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
+ pgstat_setup_memcxt();
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+ if (area)
+ return;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* stats shared memory persists for the backend lifetime */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * The first attacher backend may still reading the stats file, or the last
+ * detacher may writing it. Wait for the work to finish.
+ */
+ ConditionVariablePrepareToSleep(&StatsShmem->holdoff_cv);
+ for (;;)
+ {
+ bool hold_off;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
-
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_SHARED);
+ hold_off = StatsShmem->attach_holdoff;
+ LWLockRelease(StatsLock);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
-
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- test_byte++; /* just make sure variable is changed */
-
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /* If we get here, we have a working socket */
- break;
+ if (!hold_off)
+ break;
+
+ ConditionVariableTimedSleep(&StatsShmem->holdoff_cv, 10,
+ WAIT_EVENT_READING_STATS_FILE);
}
+ ConditionVariableCancelSleep();
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that a process going to exit can find whether it is
+ * the last one or not.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
+
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_params, 0);
+
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
+ LWLockInitialize(&StatsShmem->slru_stats.lock, LWTRANCHE_STATS);
+ pg_atomic_init_u32(&StatsShmem->slru_stats.changecount, 0);
+
+ /* Block the next attacher for a while, see the comment above. */
+ StatsShmem->attach_holdoff = true;
+
+ StatsShmem->refcount = 1;
}
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
+ LWLockRelease(StatsLock);
+
+ if (area)
{
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- ereport(LOG,
- (errmsg("getsockopt(%s) failed: %m", "SO_RCVBUF")));
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- ereport(LOG,
- (errmsg("setsockopt(%s) failed: %m", "SO_RCVBUF")));
- }
+ /*
+ * We're the first attacher process, read stats file while blocking
+ * successors.
+ */
+ Assert(StatsShmem->attach_holdoff);
+ pgstat_read_statsfile();
+ allow_next_attacher();
+ }
+ else
+ {
+ /* We're not the first one, attach existing shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_params,
+ StatsShmem->hash_handle, 0);
}
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+}
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
+/* ----------
+ * cleanup_dropped_stats_entries() -
+ * Clean up shared stats entries no longer used.
+ *
+ * Shared stats entries for dropped objects may be left referenced. Clean up
+ * our reference and drop the shared entry if needed.
+ * ----------
+ */
+static void
+cleanup_dropped_stats_entries(void)
+{
+ pgstat_localhash_iterator i;
+ PgStatLocalHashEntry *ent;
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+ if (pgStatEntHash == NULL)
+ return;
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
+ pgstat_localhash_start_iterate(pgStatEntHash, &i);
+ while ((ent = pgstat_localhash_iterate(pgStatEntHash, &i))
+ != NULL)
+ {
+ /*
+ * Free the shared memory chunk for the entry if we were the last
+ * referrer to a dropped entry.
+ */
+ if (pg_atomic_sub_fetch_u32(&ent->body->refcount, 1) < 1 &&
+ ent->body->dropped)
+ dsa_free(area, ent->dsapointer);
+ }
/*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
+ * This function is expected to be called during backend exit. So we don't
+ * bother destroying pgStatEntHash.
*/
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ pgStatEntHash = NULL;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_file)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ bool is_last_detacher = 0;
+
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
+
+ /* We shouldn't leave a reference to shared stats. */
+ cleanup_dropped_stats_entries();
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
+ /*
+ * If we are the last detacher, hold off the next attacher (if possible)
+ * until we finish writing stats file.
+ */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount == 0)
{
- int nchars;
- Oid tmp_oid;
+ StatsShmem->attach_holdoff = true;
+ is_last_detacher = true;
+ }
+ LWLockRelease(StatsLock);
- /*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
- */
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
-
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ if (is_last_detacher)
+ {
+ if (write_file)
+ pgstat_write_statsfile();
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ /* allow the next attacher, if any */
+ allow_next_attacher();
}
- FreeDir(dir);
+
+ /*
+ * Detach the area. It is automatically destroyed when the last process
+ * detached it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+
+ /* We are going to exit. Don't bother destroying local hashes. */
+ pgStatLocalHash = NULL;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
+
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
+
+ /*
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
+ */
+ detach_shared_stats(false);
+ attach_shared_stats();
}
-#ifdef EXEC_BACKEND
/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
+ * fetch_lock_statentry - common helper function to fetch and lock a stats
+ * entry for flush_tabstat, flush_funcstat and flush_dbstat.
*/
-static pid_t
-pgstat_forkexec(void)
+static PgStat_StatEntryHeader *
+fetch_lock_statentry(PgStatTypes type, Oid dboid, Oid objid, bool nowait)
{
- char *av[10];
- int ac = 0;
+ PgStat_StatEntryHeader *header;
+
+ /* find shared table stats entry corresponding to the local entry */
+ header = (PgStat_StatEntryHeader *)
+ get_stat_entry(type, dboid, objid, nowait, true, NULL);
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ /* skip if dshash failed to acquire lock */
+ if (header == NULL)
+ return false;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ return false;
- return postmaster_forkexec(ac, av);
+ return header;
}
-#endif /* EXEC_BACKEND */
-/*
- * pgstat_start() -
+/* ----------
+ * get_stat_entry() -
*
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new base entry. If found is not NULL, it is set to true
+ * if existing entry is found or false if not.
+ * ----------
+ */
+static PgStat_StatEntryHeader *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait, bool create,
+ bool *found)
+{
+ PgStatHashEntry *shhashent;
+ PgStatLocalHashEntry *lohashent;
+ PgStat_StatEntryHeader *shheader = NULL;
+ PgStatHashKey key;
+ bool shfound;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ if (pgStatEntHash)
+ {
+ uint64 currage;
+
+ /*
+ * pgStatEntHashAge increments quite slowly than the time the following
+ * loop takes so this is expected to iterate no more than twice.
+ */
+ while (unlikely
+ (pgStatEntHashAge !=
+ (currage = pg_atomic_read_u64(&StatsShmem->gc_count))))
+ {
+ pgstat_localhash_iterator i;
+
+ /*
+ * Some entries have been dropped. Invalidate cache pointer to
+ * them.
+ */
+ pgstat_localhash_start_iterate(pgStatEntHash, &i);
+ while ((lohashent = pgstat_localhash_iterate(pgStatEntHash, &i))
+ != NULL)
+ {
+ PgStat_StatEntryHeader *header = lohashent->body;
+ if (header->dropped)
+ {
+ pgstat_localhash_delete(pgStatEntHash, key);
+
+ if (pg_atomic_sub_fetch_u32(&header->refcount, 1) < 1)
+ {
+ /*
+ * We're the last referrer to this entry, drop the
+ * shared entry.
+ */
+ dsa_free(area, lohashent->dsapointer);
+ }
+ }
+ }
+
+ pgStatEntHashAge = currage;
+ }
+
+ lohashent = pgstat_localhash_lookup(pgStatEntHash, key);
+
+ if (lohashent)
+ {
+ if (found)
+ *found = true;
+ return lohashent->body;
+ }
+ }
+
+ shhashent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &shfound);
+ if (shhashent)
+ {
+ if (create && !shfound)
+ {
+ /* Create new stats entry. */
+ dsa_pointer chunk = dsa_allocate0(area,
+ pgstat_sharedentsize[type]);
+
+ shheader = dsa_get_address(area, chunk);
+ LWLockInitialize(&shheader->lock, LWTRANCHE_STATS);
+ pg_atomic_init_u32(&shheader->refcount, 0);
+
+ /* Link the new entry from the hash entry. */
+ shhashent->body = chunk;
+ }
+ else
+ shheader = dsa_get_address(area, shhashent->body);
+
+ /*
+ * We expose this shared entry now. You might think that the entry can
+ * be removed by a concurrent backend, but since we are creating an
+ * stats entry, the object actually exists and used in the upper
+ * layer. Such an object cannot be dropped until the first vacuum after
+ * the current transaction ends.
+ */
+ dshash_release_lock(pgStatSharedHash, shhashent);
+
+ /* register to local hash if possible */
+ if (pgStatEntHash || pgStatCacheContext)
+ {
+ bool lofound;
+
+ if (pgStatEntHash == NULL)
+ {
+ pgStatEntHash =
+ pgstat_localhash_create(pgStatCacheContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
+ pgStatEntHashAge =
+ pg_atomic_read_u64(&StatsShmem->gc_count);
+ }
+
+ lohashent =
+ pgstat_localhash_insert(pgStatEntHash, key, &lofound);
+
+ Assert(!lofound);
+ lohashent->body = shheader;
+ lohashent->dsapointer = shhashent->body;
+
+ pg_atomic_add_fetch_u32(&shheader->refcount, 1);
+ }
+ }
+
+ if (found)
+ *found = shfound;
+
+ return shheader;
+}
+
+/*
+ * flush_walstat - flush out a local WAL stats entries
*
- * Returns PID of child process, or 0 if fail.
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
*
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Returns true if all local WAL stats are successfully flushed out.
*/
-int
-pgstat_start(void)
+static bool
+flush_walstat(bool nowait)
{
- time_t curtime;
- pid_t pgStatPid;
+ PgStat_Wal *s = &StatsShmem->wal_stats;
+ PgStat_Wal *l = &WalStats;
+ WalUsage all_zeroes = {0} PG_USED_FOR_ASSERTS_ONLY;
+
+ /*
+ * We don't update the WAL usage portion of the local WalStats
+ * elsewhere. Instead, fill in that portion with the difference of
+ * pgWalUsage since the previous call.
+ */
+ Assert(memcmp(&l->wal_usage, &all_zeroes, sizeof(WalUsage)) == 0);
+ WalUsageAccumDiff(&l->wal_usage, &pgWalUsage, &prevWalUsage);
/*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
+ * This function can be called even if nothing at all has happened. Avoid
+ * taking lock for nothing in that case.
*/
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
+ if (!walstats_pending())
+ return true;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&StatsShmem->wal_stats_lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&StatsShmem->wal_stats_lock,
+ LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ s->wal_usage.wal_records += l->wal_usage.wal_records;
+ s->wal_usage.wal_fpi += l->wal_usage.wal_fpi;
+ s->wal_usage.wal_bytes += l->wal_usage.wal_bytes;
+ s->wal_buffers_full += l->wal_buffers_full;
+ LWLockRelease(&StatsShmem->wal_stats_lock);
/*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
+ * Save the current counters for the subsequent calculation of WAL usage.
*/
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ prevWalUsage = pgWalUsage;
/*
- * Okay, fork off the collector.
+ * Clear out the statistics buffer, so it can be re-used.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
+ MemSet(&WalStats, 0, sizeof(WalStats));
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
+ return true;
+}
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+/*
+ * flush_slrustat - flush out a local SLRU stats entries
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true. Writer processes are mutually excluded
+ * using LWLock, but readers are expected to use change-count protocol to avoid
+ * interference with writers.
+ *
+ * Returns true if all local SLRU stats are successfully flushed out.
+ */
+static bool
+flush_slrustat(bool nowait)
+{
+ uint32 assert_changecount PG_USED_FOR_ASSERTS_ONLY;
+ int i;
+
+ if (!have_slrustats)
+ return true;
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&StatsShmem->slru_stats.lock,
+ LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
- PgstatCollectorMain(0, NULL);
- break;
-#endif
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_stats.changecount, 1);
+ Assert((assert_changecount & 1) == 0);
- default:
- return (int) pgStatPid;
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS ; i++)
+ {
+ PgStat_SLRUStats *sharedent = &StatsShmem->slru_stats.entry[i];
+ PgStat_SLRUStats *localent = &local_SLRUStats[i];
+
+ sharedent->blocks_zeroed += localent->blocks_zeroed;
+ sharedent->blocks_hit += localent->blocks_hit;
+ sharedent->blocks_read += localent->blocks_read;
+ sharedent->blocks_written += localent->blocks_written;
+ sharedent->blocks_exists += localent->blocks_exists;
+ sharedent->flush += localent->flush;
+ sharedent->truncate += localent->truncate;
}
- /* shouldn't get here */
- return 0;
+ /* done, clear the local entry */
+ MemSet(local_SLRUStats, 0,
+ sizeof(PgStat_SLRUStats) * SLRU_NUM_ELEMENTS);
+
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_stats.changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+
+ have_slrustats = false;
+
+ return true;
+}
+
+/* ----------
+ * delete_current_stats_entry()
+ *
+ * Deletes the given shared entry from shared stats hash. The entry must be
+ * exclusively locked.
+ * ----------
+ */
+static void
+delete_current_stats_entry(dshash_seq_status *hstat)
+{
+ dsa_pointer pdsa;
+ PgStat_StatEntryHeader *header;
+ PgStatHashEntry *ent;
+
+ ent = dshash_get_current(hstat);
+ pdsa = ent->body;
+ header = dsa_get_address(area, pdsa);
+
+ /* No one find this entry ever after. */
+ dshash_delete_current(hstat);
+
+ /*
+ * Let the referrers drop the entry if any. Refcount won't be decremented
+ * until "dropped" is set true and StatsShmem->gc_count is incremented
+ * later. So we can check refcount to set dropped without holding a
+ * lock. If no one is referring this entry, free it immediately.
+ */
+
+ if (pg_atomic_read_u32(&header->refcount) > 0)
+ header->dropped = true;
+ else
+ dsa_free(area, pdsa);
+
+ return;
}
-void
-allow_immediate_pgstat_restart(void)
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize the statsbody part of the returned
+ * memory.
+ * ----------
+ */
+static PgStat_StatEntryHeader *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- last_pgstat_start_time = 0;
+ PgStatHashKey key;
+ PgStatLocalHashEntry *entry;
+
+ if (pgStatLocalHash == NULL)
+ pgStatLocalHash = pgstat_localhash_create(pgStatCacheContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ if (create)
+ entry = pgstat_localhash_insert(pgStatLocalHash, key, found);
+ else
+ entry = pgstat_localhash_lookup(pgStatLocalHash, key);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ entry->body = MemoryContextAllocZero(TopMemoryContext,
+ pgstat_localentsize[type]);
+
+ return entry->body;
}
/* ------------------------------------------------------------
@@ -846,150 +1138,399 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait;
int i;
+ uint64 oldval;
+
+ /* Return if not active */
+ if (area == NULL)
+ return 0;
+
+ /*
+ * We need a database entry if the following stats exists.
+ */
+ if (pgStatXactCommit > 0 || pgStatXactRollback > 0 ||
+ pgStatBlockReadTime > 0 || pgStatBlockWriteTime > 0)
+ get_local_dbstat_entry(MyDatabaseId);
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (pgStatLocalHash == NULL && have_slrustats && !walstats_pending())
+ return 0;
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
-
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
-
- /*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
- */
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+
+ if (!force)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
-
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
-
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ force = true;
+ }
+
+ /* don't wait for lock acquisition when !force */
+ nowait = !force;
+
+ if (pgStatLocalHash)
+ {
+ int remains = 0;
+ pgstat_localhash_iterator i;
+ List *dbentlist = NIL;
+ ListCell *lc;
+ PgStatLocalHashEntry *lent;
+
+ /* Step 1: flush out other than database stats */
+ pgstat_localhash_start_iterate(pgStatLocalHash, &i);
+ while ((lent = pgstat_localhash_iterate(pgStatLocalHash, &i)) != NULL)
+ {
+ bool remove = false;
+
+ switch (lent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /*
+ * flush_tabstat applies some of stats numbers of flushed
+ * entries into local database stats. Just remember the
+ * database entries for now then flush-out them later.
+ */
+ dbentlist = lappend(dbentlist, lent);
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_REPLSLOT:
+ /* We don't have that kind of local entry */
+ Assert(false);
+ }
+
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->body);
+ lent->body = NULL;
+ pgstat_localhash_delete(pgStatLocalHash, lent->key);
+ }
+
+ /* Step 2: flush out database stats */
+ foreach(lc, dbentlist)
+ {
+ PgStatLocalHashEntry *lent = (PgStatLocalHashEntry *) lfirst(lc);
+
+ if (flush_dbstat(lent, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->body);
+ lent->body = NULL;
+ pgstat_localhash_delete(pgStatLocalHash, lent->key);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ list_free(dbentlist);
+ dbentlist = NULL;
+
+ if (remains <= 0)
+ {
+ pgstat_localhash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
+ }
+
+ /* flush wal stats */
+ flush_walstat(nowait);
+
+ /* flush SLRU stats */
+ flush_slrustat(nowait);
+
+ /*
+ * Publish the time of the last flush, but we don't notify the change of
+ * the timestamp itself. Readers will get sufficiently recent timestamp.
+ * If we failed to update the value, concurrent processes should have
+ * updated it to sufficiently recent time.
+ *
+ * XXX: The loop might be unnecessary for the reason above.
+ */
+ oldval = pg_atomic_read_u64(&StatsShmem->stats_timestamp);
+
+ for (i = 0 ; i < 10 ; i++)
+ {
+ if (oldval >= now ||
+ pg_atomic_compare_exchange_u64(&StatsShmem->stats_timestamp,
+ &oldval, (uint64) now))
+ break;
}
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * Some of the local stats may have not been flushed due to lock
+ * contention. If we have such pending local stats here, let the caller
+ * know the retry interval.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
+ if (pgStatLocalHash != NULL || have_slrustats || walstats_pending())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
- /* Send WAL statistics */
- pgstat_send_wal();
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
- /* Finally send SLRU statistics */
- pgstat_send_slru();
+ return retry_interval / 1000;
+ }
+
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
}
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
*/
-static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+static bool
+flush_tabstat(PgStatLocalHashEntry *ent, bool nowait)
{
- int n;
- int len;
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+
+ Assert(ent->key.type == PGSTAT_TYPE_TABLE);
+ lstats = (PgStat_TableStatus *) ent->body;
+ dboid = ent->key.databaseid;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ {
+ /* This local entry is going to be dropped, delink from relcache. */
+ pgstat_delinkstats(lstats->relation);
+ return true;
+ }
+
+ /* find shared table stats entry corresponding to the local entry */
+ shtabstats = (PgStat_StatTabEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_TABLE, dboid, ent->key.objectid,
+ nowait);
+
+ if (shtabstats == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->inserts_since_vacuum += lstats->t_counts.t_tuples_inserted;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shtabstats->header.lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /* This local entry is going to be dropped, delink from relcache. */
+ pgstat_delinkstats(lstats->relation);
+
+ return true;
+}
+
+/*
+ * flush_funcstat - flush out a local function stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatLocalHashEntry *ent, bool nowait)
+{
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStat_StatFuncEntry *shfuncent = NULL; /* shared stats entry */
+
+ Assert(ent->key.type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) ent->body;
+
+ /* localent always has non-zero content */
+
+ /* find shared table stats entry corresponding to the local entry */
+ shfuncent = (PgStat_StatFuncEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ ent->key.objectid, nowait);
+ if (shfuncent == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ shfuncent->f_numcalls += localent->f_counts.f_numcalls;
+ shfuncent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ shfuncent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shfuncent->header.lock);
+
+ return true;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatLocalHashEntry *ent, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(ent->key.type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &ent->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ sharedent = (PgStat_StatDBEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_DB, ent->key.databaseid, InvalidOid,
+ nowait);
+
+ if (!sharedent)
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(ent->key.databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -997,281 +1538,138 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&sharedent->header.lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
-/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
- */
-static void
-pgstat_send_funcstats(void)
-{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
- {
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
-
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
-}
-
-
/* ----------
* pgstat_vacuum_stat() -
*
- * Will tell the collector about objects he can get rid of.
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
* ----------
*/
void
pgstat_vacuum_stat(void)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
+ pgstat_oid_hash *dbids; /* database ids */
+ pgstat_oid_hash *relids; /* relation ids in the current database */
+ pgstat_oid_hash *funcids;/* function ids in the current database */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
return;
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
+ nvictims = 0;
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ /* some of the dshash entries are to be removed, take exclusive lock. */
+ dshash_seq_init(&dshstat, pgStatSharedHash, true);
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
+ pgstat_oid_hash *oidhash;
+ Oid key;
CHECK_FOR_INTERRUPTS();
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
+ switch (ent->key.type)
{
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidhash = dbids;
+ key = ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidhash = relids;
+ key = ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidhash = funcids;
+ key = ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_REPLSLOT:
+ /*
+ * We don't bother vacuumming this kind of entries because the
+ * number of entries is quite small and entries are likely to
+ * be reused soon.
+ */
continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
}
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
+ /* Skip existent objects. */
+ if (pgstat_oid_lookup(oidhash, key) != NULL)
+ continue;
- hash_destroy(htab);
+ /* drop this etnry */
+ delete_current_stats_entry(&dshstat);
+ nvictims++;
}
+ dshash_seq_term(&dshstat);
+ pgstat_oid_destroy(dbids);
+ pgstat_oid_destroy(relids);
+ pgstat_oid_destroy(funcids);
+
+ if (nvictims > 0)
+ pg_atomic_add_fetch_u64(&StatsShmem->gc_count, 1);
}
-
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
- * into a temporary hash table. Caller should hash_destroy the result
+ * into a temporary hash table. Caller should pgsstat_oid_destroy the result
* when done with it. (However, we make the table in CurrentMemoryContext
* so that it will be freed properly in event of an error.)
* ----------
*/
-static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+static pgstat_oid_hash *
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
- HTAB *htab;
- HASHCTL hash_ctl;
+ pgstat_oid_hash *rethash;
Relation rel;
TableScanDesc scan;
HeapTuple tup;
Snapshot snapshot;
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(Oid);
- hash_ctl.hcxt = CurrentMemoryContext;
- htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ rethash = pgstat_oid_create(CurrentMemoryContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
rel = table_open(catalogid, AccessShareLock);
snapshot = RegisterSnapshot(GetLatestSnapshot());
@@ -1280,81 +1678,61 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
{
Oid thisoid;
bool isnull;
+ bool found;
thisoid = heap_getattr(tup, anum_oid, RelationGetDescr(rel), &isnull);
Assert(!isnull);
CHECK_FOR_INTERRUPTS();
- (void) hash_search(htab, (void *) &thisoid, HASH_ENTER, NULL);
+ pgstat_oid_insert(rethash, thisoid, &found);
}
table_endscan(scan);
UnregisterSnapshot(snapshot);
table_close(rel, AccessShareLock);
- return htab;
+ return rethash;
}
-
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ Assert(OidIsValid(databaseid));
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
-
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ /* some of the dshash entries are to be removed, take exclusive lock. */
+ dshash_seq_init(&hstat, pgStatSharedHash, true);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.databaseid == MyDatabaseId)
+ delete_current_stats_entry(&hstat);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Let readers run a garbage collection of local hashes */
+ pg_atomic_add_fetch_u64(&StatsShmem->gc_count, 1);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1363,53 +1741,146 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatEntryHeader *header;
+
+ if (p->key.databaseid != MyDatabaseId)
+ continue;
+
+ header = dsa_get_address(area, p->body);
+
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ memset(PGSTAT_SHENT_BODY(header), 0,
+ PGSTAT_SHENT_BODY_LEN(p->key.type));
+
+ if (p->key.type == PGSTAT_TYPE_DB)
+ {
+ PgStat_StatDBEntry *dbstat = (PgStat_StatDBEntry *) header;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ }
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Invalidate the simple cache keys */
+ cached_dbent_key = stathashkey_zero;
+ cached_tabent_key = stathashkey_zero;
+ cached_funcent_key = stathashkey_zero;
+}
+
+/*
+ * pgstat_copy_global_stats - helper function for functions
+ * pgstat_fetch_stat_*() and pgstat_reset_shared_counters().
+ *
+ * Copies out the specified memory area following change-count protocol.
+ */
+static inline void
+pgstat_copy_global_stats(void *dst, void*src, size_t len,
+ pg_atomic_uint32 *count)
+{
+ int before_changecount;
+ int after_changecount;
+
+ after_changecount = pg_atomic_read_u32(count);
+
+ do
+ {
+ before_changecount = after_changecount;
+ memcpy(dst, src, len);
+ after_changecount = pg_atomic_read_u32(count);
+ } while ((before_changecount & 1) == 1 ||
+ after_changecount != before_changecount);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
+ *
+ * We don't scribble on shared stats while resetting to avoid locking on
+ * shared stats struct. Instead, just record the current counters in another
+ * shared struct, which is protected by StatsLock. See
+ * pgstat_fetch_stat_(archiver|bgwriter|checkpointer) for the reader side.
* ----------
*/
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ TimestampTz now = GetCurrentTimestamp();
+ PgStat_Shared_Reset_Target t;
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ t = RESET_ARCHIVER;
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ t = RESET_BGWRITER;
else if (strcmp(target, "wal") == 0)
- msg.m_resettarget = RESET_WAL;
+ t = RESET_WAL;
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\", \"bgwriter\" or \"wal\".")));
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
+ /* Reset statistics for the cluster. */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ switch (t)
+ {
+ case RESET_ARCHIVER:
+ pgstat_copy_global_stats(&StatsShmem->archiver_reset_offset,
+ &StatsShmem->archiver_stats,
+ sizeof(PgStat_Archiver),
+ &StatsShmem->archiver_changecount);
+ StatsShmem->archiver_reset_offset.stat_reset_timestamp = now;
+ cached_archiverstats_is_valid = false;
+ break;
+
+ case RESET_BGWRITER:
+ pgstat_copy_global_stats(&StatsShmem->bgwriter_reset_offset,
+ &StatsShmem->bgwriter_stats,
+ sizeof(PgStat_BgWriter),
+ &StatsShmem->bgwriter_changecount);
+ pgstat_copy_global_stats(&StatsShmem->checkpointer_reset_offset,
+ &StatsShmem->checkpointer_stats,
+ sizeof(PgStat_CheckPointer),
+ &StatsShmem->checkpointer_changecount);
+ StatsShmem->bgwriter_reset_offset.stat_reset_timestamp = now;
+ cached_bgwriterstats_is_valid = false;
+ cached_checkpointerstats_is_valid = false;
+ break;
+
+ case RESET_WAL:
+ /*
+ * Differntly from the two above, WAL statistics has many writer
+ * processes and protected by wal_stats_lock.
+ */
+ LWLockAcquire(&StatsShmem->wal_stats_lock, LW_EXCLUSIVE);
+ MemSet(&StatsShmem->wal_stats, 0, sizeof(PgStat_Wal));
+ StatsShmem->wal_stats.stat_reset_timestamp = now;
+ LWLockRelease(&StatsShmem->wal_stats_lock);
+ cached_walstats_is_valid = false;
+ break;
+ }
+
+ LWLockRelease(StatsLock);
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1418,17 +1889,37 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStat_StatEntryHeader *header;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ dbentry = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid, false, false,
+ NULL);
+ Assert(dbentry);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&dbentry->header.lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&dbentry->header.lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ header = get_stat_entry(stattype, MyDatabaseId, objoid, false, false, NULL);
+
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ memset(PGSTAT_SHENT_BODY(header), 0, PGSTAT_SHENT_BODY_LEN(stattype));
+ LWLockRelease(&header->lock);
}
/* ----------
@@ -1444,15 +1935,40 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_reset_slru_counter(const char *name)
{
- PgStat_MsgResetslrucounter msg;
+ int i;
+ TimestampTz ts = GetCurrentTimestamp();
+ uint32 assert_changecount;PG_USED_FOR_ASSERTS_ONLY;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ if (name)
+ {
+ i = pgstat_slru_index(name);
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_changecount, 1);
+ Assert((assert_changecount & 1) == 0);
+ MemSet(&StatsShmem->slru_stats.entry[i], 0,
+ sizeof(PgStat_SLRUStats));
+ StatsShmem->slru_stats.entry[i].stat_reset_timestamp = ts;
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+ }
+ else
+ {
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_changecount, 1);
+ Assert((assert_changecount & 1) == 0);
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS; i++)
+ {
+ MemSet(&StatsShmem->slru_stats.entry[i], 0,
+ sizeof(PgStat_SLRUStats));
+ StatsShmem->slru_stats.entry[i].stat_reset_timestamp = ts;
+ }
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSLRUCOUNTER);
- msg.m_index = (name) ? pgstat_slru_index(name) : -1;
-
- pgstat_send(&msg, sizeof(msg));
+ cached_slrustats_is_valid = false;
}
/* ----------
@@ -1468,20 +1984,19 @@ pgstat_reset_slru_counter(const char *name)
void
pgstat_reset_replslot_counter(const char *name)
{
- PgStat_MsgResetreplslotcounter msg;
+ int startidx;
+ int endidx;
+ int i;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
if (name)
{
ReplicationSlot *slot;
-
- /*
- * Check if the slot exits with the given name. It is possible that by
- * the time this message is executed the slot is dropped but at least
- * this check will ensure that the given name is for a valid slot.
- */
+
+ /* Check if the slot exits with the given name. */
LWLockAcquire(ReplicationSlotControlLock, LW_SHARED);
slot = SearchNamedReplicationSlot(name);
LWLockRelease(ReplicationSlotControlLock);
@@ -1499,15 +2014,36 @@ pgstat_reset_replslot_counter(const char *name)
if (SlotIsPhysical(slot))
return;
- strlcpy(msg.m_slotname, name, NAMEDATALEN);
- msg.clearall = false;
+ /* reset this one entry */
+ startidx = endidx = slot - ReplicationSlotCtl->replication_slots;
}
else
- msg.clearall = true;
+ {
+ /* reset all existent entries */
+ startidx = 0;
+ endidx = max_replication_slots - 1;
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETREPLSLOTCOUNTER);
+ ts = GetCurrentTimestamp();
+ for (i = startidx ; i <= endidx ; i++)
+ {
+ PgStat_ReplSlot *shent;
- pgstat_send(&msg, sizeof(msg));
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, false, NULL);
+
+ /* Skip non-existent entries */
+ if (!shent)
+ continue;
+
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ memset(&shent->spill_txns, 0,
+ offsetof(PgStat_ReplSlot, stat_reset_timestamp) -
+ offsetof(PgStat_ReplSlot, spill_txns));
+ shent->stat_reset_timestamp = ts;
+ LWLockRelease(&shent->header.lock);
+ }
}
/* ----------
@@ -1521,48 +2057,93 @@ pgstat_reset_replslot_counter(const char *name)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ /*
+ * End-of-vacuum is reported instantly. Report the start the same way for
+ * consistency. Vacuum doesn't run frequently and is a long-lasting
+ * operation so it doesn't matter if we get blocked here a little.
+ */
+ dbentry = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, dboid, InvalidOid, false, true, NULL);
- pgstat_send(&msg, sizeof(msg));
+ ts = GetCurrentTimestamp();;
+ LWLockAcquire(&dbentry->header.lock, LW_EXCLUSIVE);
+ dbentry->last_autovac_time = ts;
+ LWLockRelease(&dbentry->header.lock);
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+
+ /*
+ * Differently from ordinary operations, maintenance commands take longer
+ * time and getting blocked at the end of work doesn't matter. Furthermore,
+ * this can prevent the stats updates made by the transactions that ends
+ * after this vacuum from being canceled by a delayed vacuum report.
+ * Update shared stats entry directly for the above reasons.
+ */
+ tabentry = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, tableoid, false, true, NULL);
+
+ LWLockAcquire(&tabentry->header.lock, LW_EXCLUSIVE);
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * It is quite possible that a non-aggressive VACUUM ended up skipping
+ * various pages, however, we'll zero the insert counter here regardless.
+ * It's currently used only to track when we need to perform an "insert"
+ * autovacuum, which are mainly intended to freeze newly inserted tuples.
+ * Zeroing this may just mean we'll not try to vacuum the table again
+ * until enough tuples have been inserted to trigger another insert
+ * autovacuum. An anti-wraparound autovacuum will catch any persistent
+ * stragglers.
+ */
+ tabentry->inserts_since_vacuum = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->vacuum_count++;
+ }
+
+ LWLockRelease(&tabentry->header.lock);
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1573,9 +2154,11 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId);
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1583,10 +2166,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1604,137 +2187,223 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Differently from ordinary operations, maintenance commands take longer
+ * time and getting blocked at the end of work doesn't matter. Furthermore,
+ * this can prevent the stats updates made by the transactions that ends
+ * after this analyze from being canceled by a delayed analyze report.
+ * Update shared stats entry directly for the above reasons.
+ */
+ tabentry = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, RelationGetRelid(rel),
+ false, true, NULL);
+
+ LWLockAcquire(&tabentry->header.lock, LW_EXCLUSIVE);
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+ LWLockRelease(&tabentry->header.lock);
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
+
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
-
-
/* --------
- * pgstat_report_checksum_failures_in_db() -
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Tell the collector about one or more checksum failures.
+ * Reports about one or more checksum failures.
* --------
*/
void
pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgChecksumFailure msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
+ dbentry = get_local_dbstat_entry(dboid);
- pgstat_send(&msg, sizeof(msg));
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
/* ----------
* pgstat_report_replslot() -
*
- * Tell the collector about replication slot statistics.
+ * Report replication slot activity.
* ----------
*/
void
-pgstat_report_replslot(const char *slotname, int spilltxns, int spillcount,
- int spillbytes, int streamtxns, int streamcount, int streambytes)
+pgstat_report_replslot(const char *slotname,
+ int spilltxns, int spillcount, int spillbytes,
+ int streamtxns, int streamcount, int streambytes)
{
- PgStat_MsgReplSlot msg;
+ PgStat_ReplSlot *shent;
+ int i;
+ bool found;
+
+ if (!area)
+ return;
+
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ if (strcmp(NameStr(ReplicationSlotCtl->replication_slots[i].data.name),
+ slotname) == 0)
+ break;
+
+ }
/*
- * Prepare and send the message
+ * the slot should have been removed. just ignore it. We create the entry
+ * for the slot with this name next time.
*/
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_REPLSLOT);
- strlcpy(msg.m_slotname, slotname, NAMEDATALEN);
- msg.m_drop = false;
- msg.m_spill_txns = spilltxns;
- msg.m_spill_count = spillcount;
- msg.m_spill_bytes = spillbytes;
- msg.m_stream_txns = streamtxns;
- msg.m_stream_count = streamcount;
- msg.m_stream_bytes = streambytes;
- pgstat_send(&msg, sizeof(PgStat_MsgReplSlot));
+ if (i == max_replication_slots)
+ return;
+
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, true, &found);
+
+ /* Clear the counters and reset dropped when we reuse it */
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ if (shent->header.dropped || !found)
+ {
+ memset(&shent->spill_txns, 0,
+ sizeof(PgStat_ReplSlot) - offsetof(PgStat_ReplSlot, spill_txns));
+ strlcpy(shent->slotname, slotname, NAMEDATALEN);
+ shent->header.dropped = false;
+ }
+
+ shent->spill_txns += spilltxns;
+ shent->spill_count += spillcount;
+ shent->spill_bytes += spillbytes;
+ shent->stream_txns += streamtxns;
+ shent->stream_count += streamcount;
+ shent->stream_bytes += streambytes;
+ LWLockRelease(&shent->header.lock);
}
/* ----------
@@ -1746,55 +2415,44 @@ pgstat_report_replslot(const char *slotname, int spilltxns, int spillcount,
void
pgstat_report_replslot_drop(const char *slotname)
{
- PgStat_MsgReplSlot msg;
+ int i;
+ PgStat_ReplSlot *shent;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_REPLSLOT);
- strlcpy(msg.m_slotname, slotname, NAMEDATALEN);
- msg.m_drop = true;
- pgstat_send(&msg, sizeof(PgStat_MsgReplSlot));
-}
+ Assert(area);
+ if (!area)
+ return;
-/* ----------
- * pgstat_ping() -
- *
- * Send some junk data to the collector to increase traffic.
- * ----------
- */
-void
-pgstat_ping(void)
-{
- PgStat_MsgDummy msg;
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ if (strcmp(NameStr(ReplicationSlotCtl->replication_slots[i].data.name),
+ slotname) == 0)
+ break;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ /* XXX: maybe the slot has been removed. just ignore it. */
+ if (i == max_replication_slots)
+ return;
+
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, false, NULL);
+
+ if (shent && !shent->header.dropped)
+ {
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ shent->header.dropped = true;
+ LWLockRelease(&shent->header.lock);
+ }
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
@@ -1809,24 +2467,9 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
- if (!found)
- MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent = (PgStat_BackendFunctionEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
fcu->fs = &htabent->f_counts;
@@ -1840,31 +2483,37 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
- return NULL;
+ PgStat_BackendFunctionEntry *ent;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ ent = (PgStat_BackendFunctionEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+
+ return ent;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1905,9 +2554,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1919,8 +2565,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1936,7 +2581,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1947,120 +2593,60 @@ pgstat_initstats(Relation rel)
* If we already set up this relation in the current transaction, nothing
* to do.
*/
- if (rel->pgstat_info != NULL &&
- rel->pgstat_info->t_id == rel_id)
+ if (rel->pgstat_info != NULL)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ /* mark this relation as the owner */
+
+ /* don't allow link a stats to multiple relcache entries */
+ Assert (rel->pgstat_info->relation == NULL);
+ rel->pgstat_info->relation = rel;
}
/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+ * pgstat_delinkstats() -
+ *
+ * Break the mutual link between a relcache entry and a local stats entry.
+ * This must be called always when one end of the link is removed.
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+void
+pgstat_delinkstats(Relation rel)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
-
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ /* remove the link to stats info if any */
+ if (rel && rel->pgstat_info)
{
- HASHCTL ctl;
-
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ /* ilnk sanity check */
+ Assert (rel->pgstat_info->relation == rel);
+ rel->pgstat_info->relation = NULL;
+ rel->pgstat_info = NULL;
}
-
- /*
- * Find an entry or create a new one.
- */
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
- if (!found)
- {
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
- }
-
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
-
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
-
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
-
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
-
- return entry;
}
/*
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel_id in the current
+ * database. If not found, try finding from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry found, return NULL, don't create a new one
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStat_TableStatus *ent;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ ent = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!ent)
+ ent = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return ent;
}
/*
@@ -2475,8 +3061,6 @@ AtPrepare_PgStat(void)
record.inserted_pre_trunc = trans->inserted_pre_trunc;
record.updated_pre_trunc = trans->updated_pre_trunc;
record.deleted_pre_trunc = trans->deleted_pre_trunc;
- record.t_id = tabstat->t_id;
- record.t_shared = tabstat->t_shared;
record.t_truncated = trans->truncated;
RegisterTwoPhaseRecord(TWOPHASE_RM_PGSTAT_ID, 0,
@@ -2491,8 +3075,8 @@ AtPrepare_PgStat(void)
*
* All we need do here is unlink the transaction stats state from the
* nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
+ * reported to the activity stats facility immediately, while the effects on
+ * live and dead tuple counts are preserved in the 2PC state file.
*
* Note: AtEOXact_PgStat is not called during PREPARE.
*/
@@ -2537,7 +3121,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2573,7 +3157,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2593,85 +3177,138 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends in a palloc'ed memory.
+ *
+ * The returned entry is stored in static memory so the content is valid until
+ * the next call of the same function for the different database.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ PgStat_StatDBEntry *shent;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for InvalidOid */
+ Assert(dbid != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_dbent_key.databaseid == dbid)
+ return &cached_dbent;
+
+ shent = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid, true, false, NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_dbent, shent, sizeof(PgStat_StatDBEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_dbent_key.databaseid = dbid;
+
+ return &cached_dbent;
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_extended(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_extended(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_extended() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is stored
+ * in static memory so the content is valid until the next call of the same
+ * function for the different table.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_extended(bool shared, Oid reloid)
+{
+ PgStat_StatTabEntry *shent;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for the InvalidOid */
+ Assert(reloid != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_tabent_key.databaseid == dboid &&
+ cached_tabent_key.objectid == reloid)
+ return &cached_tabent;
+
+ shent = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, reloid, true, false, NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_tabent, shent, sizeof(PgStat_StatTabEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_tabent_key.databaseid = dboid;
+ cached_tabent_key.objectid = reloid;
+
+ return &cached_tabent;
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2680,30 +3317,46 @@ pgstat_fetch_stat_tabentry(Oid relid)
*
* Support function for the SQL-callable pgstat* functions. Returns
* the collected statistics for one function or NULL.
+ *
+ * The returned entry is stored in static memory so the content is valid until
+ * the next call of the same function for the different function id.
* ----------
*/
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
-
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
-
- return funcentry;
+ PgStat_StatFuncEntry *shent;
+ Oid dboid = MyDatabaseId;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for the InvalidOid */
+ Assert(func_id != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_funcent_key.databaseid == dboid &&
+ cached_funcent_key.objectid == func_id)
+ return &cached_funcent;
+
+ shent = (PgStat_StatFuncEntry *)
+ get_stat_entry(PGSTAT_TYPE_FUNCTION, dboid, func_id, true, false,
+ NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_funcent, shent, sizeof(PgStat_StatFuncEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_funcent_key.databaseid = dboid;
+ cached_funcent_key.objectid = func_id;
+
+ return &cached_funcent;
}
-
/* ----------
* pgstat_fetch_stat_beentry() -
*
@@ -2763,53 +3416,160 @@ pgstat_fetch_stat_numbackends(void)
return localNumBackends;
}
+/*
+ * ---------
+ * pgstat_get_stat_timestamp() -
+ *
+ * Returns the last update timstamp of global staticstics.
+ */
+TimestampTz
+pgstat_get_stat_timestamp(void)
+{
+ return (TimestampTz) pg_atomic_read_u64(&StatsShmem->stats_timestamp);
+}
+
/*
* ---------
* pgstat_fetch_stat_archiver() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the archiver statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_ArchiverStats *
+PgStat_Archiver *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ PgStat_Archiver reset;
+ PgStat_Archiver *reset_shared = &StatsShmem->archiver_reset_offset;
+ PgStat_Archiver *shared = &StatsShmem->archiver_stats;
+ PgStat_Archiver *cached = &cached_archiverstats;
- return &archiverStats;
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_Archiver),
+ &StatsShmem->archiver_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_Archiver));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ if (cached->archived_count == reset.archived_count)
+ {
+ cached->last_archived_wal[0] = 0;
+ cached->last_archived_timestamp = 0;
+ }
+ cached->archived_count -= reset.archived_count;
+
+ if (cached->failed_count == reset.failed_count)
+ {
+ cached->last_failed_wal[0] = 0;
+ cached->last_failed_timestamp = 0;
+ }
+ cached->failed_count -= reset.failed_count;
+
+ cached->stat_reset_timestamp = reset.stat_reset_timestamp;
+
+ cached_archiverstats_is_valid = true;
+
+ return &cached_archiverstats;
}
/*
* ---------
- * pgstat_fetch_global() -
+ * pgstat_fetch_stat_bgwriter() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the global statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_GlobalStats *
-pgstat_fetch_global(void)
+PgStat_BgWriter *
+pgstat_fetch_stat_bgwriter(void)
{
- backend_read_statsfile();
+ PgStat_BgWriter reset;
+ PgStat_BgWriter *reset_shared = &StatsShmem->bgwriter_reset_offset;
+ PgStat_BgWriter *shared = &StatsShmem->bgwriter_stats;
+ PgStat_BgWriter *cached = &cached_bgwriterstats;
+
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_BgWriter),
+ &StatsShmem->bgwriter_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_BgWriter));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ cached->buf_written_clean -= reset.buf_written_clean;
+ cached->maxwritten_clean -= reset.maxwritten_clean;
+ cached->buf_alloc -= reset.buf_alloc;
+ cached->stat_reset_timestamp = reset.stat_reset_timestamp;
+
+ cached_bgwriterstats_is_valid = true;
+
+ return &cached_bgwriterstats;
+}
+
+/*
+ * ---------
+ * pgstat_fetch_stat_checkpinter() -
+ *
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
+ * ---------
+ */
+PgStat_CheckPointer *
+pgstat_fetch_stat_checkpointer(void)
+{
+ PgStat_CheckPointer reset;
+ PgStat_CheckPointer *reset_shared = &StatsShmem->checkpointer_reset_offset;
+ PgStat_CheckPointer *shared = &StatsShmem->checkpointer_stats;
+ PgStat_CheckPointer *cached = &cached_checkpointerstats;
+
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_CheckPointer),
+ &StatsShmem->checkpointer_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_CheckPointer));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ cached->timed_checkpoints -= reset.timed_checkpoints;
+ cached->requested_checkpoints -= reset.requested_checkpoints;
+ cached->buf_written_checkpoints -= reset.buf_written_checkpoints;
+ cached->buf_written_backend -= reset.buf_written_backend;
+ cached->buf_fsync_backend -= reset.buf_fsync_backend;
+ cached->checkpoint_write_time -= reset.checkpoint_write_time;
+ cached->checkpoint_sync_time -= reset.checkpoint_sync_time;
+
+ cached_checkpointerstats_is_valid = true;
- return &globalStats;
+ return &cached_checkpointerstats;
}
/*
* ---------
* pgstat_fetch_stat_wal() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the WAL statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned entry
+ * is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_WalStats *
+PgStat_Wal *
pgstat_fetch_stat_wal(void)
{
- backend_read_statsfile();
+ if (!cached_walstats_is_valid)
+ {
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&cached_walstats, &StatsShmem->wal_stats, sizeof(PgStat_Wal));
+ LWLockRelease(StatsLock);
+ }
- return &walStats;
+ cached_walstats_is_valid = true;
+
+ return &cached_walstats;
}
/*
@@ -2823,9 +3583,27 @@ pgstat_fetch_stat_wal(void)
PgStat_SLRUStats *
pgstat_fetch_slru(void)
{
- backend_read_statsfile();
+ size_t size = sizeof(PgStat_SLRUStats) * SLRU_NUM_ELEMENTS;
- return slruStats;
+ for (;;)
+ {
+ uint32 before_count;
+ uint32 after_count;
+
+ pg_read_barrier();
+ before_count = pg_atomic_read_u32(&StatsShmem->slru_changecount);
+ memcpy(&cached_slrustats, &StatsShmem->slru_stats, size);
+ after_count = pg_atomic_read_u32(&StatsShmem->slru_changecount);
+
+ if (before_count == after_count && (before_count & 1) == 0)
+ break;
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ cached_slrustats_is_valid = true;
+
+ return &cached_slrustats;
}
/*
@@ -2837,13 +3615,41 @@ pgstat_fetch_slru(void)
* number of entries in nslots_p.
* ---------
*/
-PgStat_ReplSlotStats *
+PgStat_ReplSlot *
pgstat_fetch_replslot(int *nslots_p)
{
- backend_read_statsfile();
- *nslots_p = nReplSlotStats;
- return replSlotStats;
+ if (cached_replslotstats == NULL)
+ {
+ cached_replslotstats = (PgStat_ReplSlot *)
+ MemoryContextAlloc(pgStatCacheContext,
+ sizeof(PgStat_ReplSlot) * max_replication_slots);
+ }
+
+ if (n_cached_replslotstats < 0)
+ {
+ int n = 0;
+ int i;
+
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ PgStat_ReplSlot *shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT, MyDatabaseId, i,
+ false, false, NULL);
+ if (shent && !shent->header.dropped)
+ {
+ memcpy(cached_replslotstats[n++].slotname,
+ shent->slotname,
+ sizeof(PgStat_ReplSlot) -
+ offsetof(PgStat_ReplSlot, slotname));
+ }
+ }
+
+ n_cached_replslotstats = n;
+ }
+
+ *nslots_p = n_cached_replslotstats;
+ return cached_replslotstats;
}
/* ------------------------------------------------------------
@@ -3068,8 +3874,8 @@ pgstat_initialize(void)
*/
prevWalUsage = pgWalUsage;
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3246,12 +4052,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3264,13 +4073,22 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
if (OidIsValid(MyDatabaseId))
pgstat_report_stat(true);
+ /*
+ * We need to clean up temporary slots before detaching shared statistics
+ * so that the statistics for temporary slots are properly removed.
+ */
+ if (MyReplicationSlot != NULL)
+ ReplicationSlotRelease();
+
+ ReplicationSlotCleanup();
+
/*
* Clear my status entry, following the protocol of bumping st_changecount
* before and after. We use a volatile pointer here to ensure the
@@ -3281,6 +4099,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3541,7 +4361,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3836,8 +4657,8 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
+ case WAIT_EVENT_READING_STATS_FILE:
+ event_name = "ReadingStatsFile";
break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
@@ -4491,94 +5312,80 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
+ * Report archiver statistics
* ----------
*/
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
+void
+pgstat_report_archiver(const char *xlog, bool failed)
{
- int rc;
+ TimestampTz now = GetCurrentTimestamp();
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
- ((PgStat_MsgHdr *) msg)->m_size = len;
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->archiver_changecount, 1);
+ Assert((before_count & 1) == 0);
- /* We'll retry after EINTR, but ignore all other failures */
- do
+ if (failed)
{
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
- * ----------
- */
-void
-pgstat_send_archiver(const char *xlog, bool failed)
-{
- PgStat_MsgArchiver msg;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- strlcpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ ++StatsShmem->archiver_stats.failed_count;
+ memcpy(&StatsShmem->archiver_stats.last_failed_wal, xlog,
+ sizeof(StatsShmem->archiver_stats.last_failed_wal));
+ StatsShmem->archiver_stats.last_failed_timestamp = now;
+ }
+ else
+ {
+ ++StatsShmem->archiver_stats.archived_count;
+ memcpy(&StatsShmem->archiver_stats.last_archived_wal, xlog,
+ sizeof(StatsShmem->archiver_stats.last_archived_wal));
+ StatsShmem->archiver_stats.last_archived_timestamp = now;
+ }
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->archiver_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+ PgStat_BgWriter *s = &StatsShmem->bgwriter_stats;
+ PgStat_BgWriter *l = &BgWriterStats;
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->bgwriter_changecount, 1);
+ Assert((before_count & 1) == 0);
+
+ s->buf_written_clean += l->buf_written_clean;
+ s->maxwritten_clean += l->maxwritten_clean;
+ s->buf_alloc += l->buf_alloc;
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->bgwriter_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4587,537 +5394,136 @@ pgstat_send_bgwriter(void)
}
/* ----------
- * pgstat_send_wal() -
+ * pgstat_report_checkpointer() -
*
- * Send WAL statistics to the collector
+ * Report checkpointer statistics
* ----------
*/
void
-pgstat_send_wal(void)
+pgstat_report_checkpointer(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgWal all_zeroes;
-
- WalUsage walusage;
-
- /*
- * Calculate how much WAL usage counters are increased by substracting the
- * previous counters from the current ones. Fill the results in WAL stats
- * message.
- */
- MemSet(&walusage, 0, sizeof(WalUsage));
- WalUsageAccumDiff(&walusage, &pgWalUsage, &prevWalUsage);
-
- WalStats.m_wal_records = walusage.wal_records;
- WalStats.m_wal_fpi = walusage.wal_fpi;
- WalStats.m_wal_bytes = walusage.wal_bytes;
+ static const PgStat_CheckPointer all_zeroes;
+ PgStat_CheckPointer *s = &StatsShmem->checkpointer_stats;
+ PgStat_CheckPointer *l = &CheckPointerStats;
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&WalStats, &all_zeroes, sizeof(PgStat_MsgWal)) == 0)
+ if (memcmp(&CheckPointerStats, &all_zeroes,
+ sizeof(PgStat_CheckPointer)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&WalStats.m_hdr, PGSTAT_MTYPE_WAL);
- pgstat_send(&WalStats, sizeof(WalStats));
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->checkpointer_changecount, 1);
+ Assert((before_count & 1) == 0);
- /*
- * Save the current counters for the subsequent calculation of WAL usage.
- */
- prevWalUsage = pgWalUsage;
+ s->timed_checkpoints += l->timed_checkpoints;
+ s->requested_checkpoints += l->requested_checkpoints;
+ s->checkpoint_write_time += l->checkpoint_write_time;
+ s->checkpoint_sync_time += l->checkpoint_sync_time;
+ s->buf_written_checkpoints += l->buf_written_checkpoints;
+ s->buf_written_backend += l->buf_written_backend;
+ s->buf_fsync_backend += l->buf_fsync_backend;
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->checkpointer_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
/*
* Clear out the statistics buffer, so it can be re-used.
*/
- MemSet(&WalStats, 0, sizeof(WalStats));
+ MemSet(&CheckPointerStats, 0, sizeof(CheckPointerStats));
}
/* ----------
- * pgstat_send_slru() -
+ * pgstat_report_wal() -
*
- * Send SLRU statistics to the collector
+ * Report WAL statistics
* ----------
*/
-static void
-pgstat_send_slru(void)
+void
+pgstat_report_wal(void)
{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgSLRU all_zeroes;
-
- for (int i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /*
- * This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
- */
- if (memcmp(&SLRUStats[i], &all_zeroes, sizeof(PgStat_MsgSLRU)) == 0)
- continue;
-
- /* set the SLRU type before each send */
- SLRUStats[i].m_index = i;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&SLRUStats[i].m_hdr, PGSTAT_MTYPE_SLRU);
- pgstat_send(&SLRUStats[i], sizeof(PgStat_MsgSLRU));
-
- /*
- * Clear out the statistics buffer, so it can be re-used.
- */
- MemSet(&SLRUStats[i], 0, sizeof(PgStat_MsgSLRU));
- }
+ flush_walstat(false);
}
-
/* ----------
- * PgstatCollectorMain() -
+ * get_local_dbstat_entry() -
*
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
- WaitEvent event;
- WaitEventSet *wes;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /* Prepare to wait for our latch or data in our socket. */
- wes = CreateWaitEventSet(CurrentMemoryContext, 3);
- AddWaitEventToSet(wes, WL_LATCH_SET, PGINVALID_SOCKET, MyLatch, NULL);
- AddWaitEventToSet(wes, WL_POSTMASTER_DEATH, PGINVALID_SOCKET, NULL, NULL);
- AddWaitEventToSet(wes, WL_SOCKET_READABLE, pgStatSock, NULL, NULL);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSLRUCOUNTER:
- pgstat_recv_resetslrucounter(&msg.msg_resetslrucounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETREPLSLOTCOUNTER:
- pgstat_recv_resetreplslotcounter(&msg.msg_resetreplslotcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_WAL:
- pgstat_recv_wal(&msg.msg_wal, len);
- break;
-
- case PGSTAT_MTYPE_SLRU:
- pgstat_recv_slru(&msg.msg_slru, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- case PGSTAT_MTYPE_REPLSLOT:
- pgstat_recv_replslot(&msg.msg_replslot, len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitEventSetWait(wes, -1L, &event, 1, WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitEventSetWait(wes, 2 * 1000L /* msec */ , &event, 1,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr == 1 && event.events == WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- FreeWaitEventSet(wes);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
*/
static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
+get_local_dbstat_entry(Oid dbid)
{
- PgStat_StatDBEntry *result;
+ PgStat_StatDBEntry *dbentry;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
/*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
+ * Find an entry or create a new one.
*/
- if (!found)
- reset_dbentry_counters(result);
+ dbentry = (PgStat_StatDBEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
- return result;
+ return dbentry;
}
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
*/
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
{
- PgStat_StatTabEntry *result;
+ PgStat_TableStatus *tabentry;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
+ tabentry = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->inserts_since_vacuum = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
+ return tabentry;
}
-
/* ----------
- * pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
+ * pgstat_write_statsfile() -
+ * Write the global statistics file, as well as DB files.
*
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * This function is called in the last process that is accessing the shared
+ * stats so locking is not required.
* ----------
*/
static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+pgstat_write_statsfile(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
- int i;
+ dshash_seq_status hstat;
+ PgStatHashEntry *ps;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
+
+ /* this is the last process that is accesing the shared stats */
+#ifdef USE_ASSERT_CHECKING
+ LWLockAcquire(StatsLock, LW_SHARED);
+ Assert(StatsShmem->refcount == 0);
+ LWLockRelease(StatsLock);
+#endif
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -5137,7 +5543,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ pg_atomic_write_u64(&StatsShmem->stats_timestamp, GetCurrentTimestamp());
/*
* Write the file header --- currently just a format ID.
@@ -5147,200 +5553,72 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
/*
- * Write global stats struct
+ * Write bgwriter global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(&StatsShmem->bgwriter_stats, sizeof(PgStat_BgWriter), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Write archiver stats struct
+ * Write checkpointer global stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(&StatsShmem->checkpointer_stats, sizeof(PgStat_CheckPointer), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Write WAL stats struct
+ * Write archiver global stats struct
*/
- rc = fwrite(&walStats, sizeof(walStats), 1, fpout);
+ rc = fwrite(&StatsShmem->archiver_stats, sizeof(PgStat_Archiver), 1,
+ fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Write WAL global stats struct
+ */
+ rc = fwrite(&StatsShmem->wal_stats, sizeof(PgStat_Wal), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write SLRU stats struct
*/
- rc = fwrite(slruStats, sizeof(slruStats), 1, fpout);
+ rc = fwrite(&StatsShmem->slru_stats, sizeof(PgStatSharedSLRUStats), 1,
+ fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Walk through the database table.
+ * Walk through the stats entry
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((ps = dshash_seq_next(&hstat)) != NULL)
{
- /*
- * Write out the table and function stats for this DB into the
- * appropriate per-DB stat file, if required.
- */
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
+ PgStat_StatEntryHeader *shent;
+ size_t len;
+
+ CHECK_FOR_INTERRUPTS();
+
+ shent = (PgStat_StatEntryHeader *)dsa_get_address(area, ps->body);
+
+ /* we may have some "dropped" entries not yet removed, skip them */
+ if (shent->dropped)
+ continue;
+
+ /* Make DB's timestamp consistent with the global stats */
+ if (ps->key.type == PGSTAT_TYPE_DB)
{
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) shent;
- pgstat_write_db_statsfile(dbentry, permanent);
+ dbentry->stats_timestamp =
+ (TimestampTz) pg_atomic_read_u64(&StatsShmem->stats_timestamp);
}
- /*
- * Write out the DB entry. We don't write the tables or functions
- * pointers, since they're of no use to any other process.
- */
- fputc('D', fpout);
- rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * Write replication slot stats struct
- */
- for (i = 0; i < nReplSlotStats; i++)
- {
- fputc('R', fpout);
- rc = fwrite(&replSlotStats[i], sizeof(PgStat_ReplSlotStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
- */
- fputc('E', fpout);
-
- if (ferror(fpout))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write temporary statistics file \"%s\": %m",
- tmpfile)));
- FreeFile(fpout);
- unlink(tmpfile);
- }
- else if (FreeFile(fpout) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not close temporary statistics file \"%s\": %m",
- tmpfile)));
- unlink(tmpfile);
- }
- else if (rename(tmpfile, statfile) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
- tmpfile, statfile)));
- unlink(tmpfile);
- }
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
-}
-
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
-
-/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
- * ----------
- */
-static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
-{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpout;
- int32 format_id;
- Oid dbid = dbentry->databaseid;
- int rc;
- char tmpfile[MAXPGPATH];
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "writing stats file \"%s\"", statfile);
-
- /*
- * Open the statistics temp file to write out the current values.
- */
- fpout = AllocateFile(tmpfile, PG_BINARY_W);
- if (fpout == NULL)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not open temporary statistics file \"%s\": %m",
- tmpfile)));
- return;
- }
-
- /*
- * Write the file header --- currently just a format ID.
- */
- format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Walk through the database's access stats per table.
- */
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
- {
- fputc('T', fpout);
- rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
+ fputc('S', fpout);
+ rc = fwrite(&ps->key, sizeof(PgStatHashKey), 1, fpout);
- /*
- * Walk through the database's function stats table.
- */
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
- {
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ /* Write except the header part of the etnry */
+ len = PGSTAT_SHENT_BODY_LEN(ps->key.type);
+ rc = fwrite(PGSTAT_SHENT_BODY(shent), len, 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ dshash_seq_term(&hstat);
/*
* No more output to be done. Close the temp file and replace the old
@@ -5374,113 +5652,63 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
}
/* ----------
- * pgstat_read_statsfiles() -
- *
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * pgstat_read_statsfile() -
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
+ * Reads in existing activity statistics file into the shared stats hash.
*
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
+ * This function is called in the only process that is accessing the shared
+ * stats so locking is not required.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+static void
+pgstat_read_statsfile(void)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
- int i;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+ char tag;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ /* this is the only process that is accesing the shared stats */
+#ifdef USE_ASSERT_CHECKING
+ LWLockAcquire(StatsLock, LW_SHARED);
+ Assert(StatsShmem->refcount == 1);
+ LWLockRelease(StatsLock);
+#endif
- /* Allocate the space for replication slot statistics */
- replSlotStats = palloc0(max_replication_slots * sizeof(PgStat_ReplSlotStats));
- nReplSlotStats = 0;
-
- /*
- * Clear out global, archiver, WAL and SLRU statistics so they start from
- * zero in case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
- memset(&walStats, 0, sizeof(walStats));
- memset(&slruStats, 0, sizeof(slruStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
- walStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all SLRU items too.
- */
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- slruStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all replication slots too.
- */
- for (i = 0; i < max_replication_slots; i++)
- replSlotStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ StatsShmem->bgwriter_stats.stat_reset_timestamp = GetCurrentTimestamp();
+ StatsShmem->archiver_stats.stat_reset_timestamp =
+ StatsShmem->bgwriter_stats.stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5489,681 +5717,150 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- goto done;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
- goto done;
- }
-
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
- /*
- * Read archiver stats struct
- */
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
- goto done;
- }
-
- /*
- * Read WAL stats struct
- */
- if (fread(&walStats, 1, sizeof(walStats), fpin) != sizeof(walStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&walStats, 0, sizeof(walStats));
goto done;
}
/*
- * Read SLRU stats struct
+ * Read bgwiter stats struct
*/
- if (fread(slruStats, 1, sizeof(slruStats), fpin) != sizeof(slruStats))
+ if (fread(&StatsShmem->bgwriter_stats, 1, sizeof(PgStat_BgWriter), fpin) !=
+ sizeof(PgStat_BgWriter))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&slruStats, 0, sizeof(slruStats));
+ MemSet(&StatsShmem->bgwriter_stats, 0, sizeof(PgStat_BgWriter));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Add to the DB hash
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
-
- break;
-
- /*
- * 'R' A PgStat_ReplSlotStats struct describing a replication
- * slot follows.
- */
- case 'R':
- if (fread(&replSlotStats[nReplSlotStats], 1, sizeof(PgStat_ReplSlotStats), fpin)
- != sizeof(PgStat_ReplSlotStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- memset(&replSlotStats[nReplSlotStats], 0, sizeof(PgStat_ReplSlotStats));
- goto done;
- }
- nReplSlotStats++;
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-
- return dbhash;
-}
-
-
-/* ----------
- * pgstat_read_db_statsfile() -
- *
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
- * ----------
- */
-static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
-{
- PgStat_StatTabEntry *tabentry;
- PgStat_StatTabEntry tabbuf;
- PgStat_StatFuncEntry funcbuf;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpin;
- int32 format_id;
- bool found;
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
-
- /*
- * Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
- *
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
- * condition is suspicious.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return;
- }
-
- /*
- * Verify it's of the expected format.
+ * Read checkpointer stats struct
*/
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
+ if (fread(&StatsShmem->checkpointer_stats, 1, sizeof(PgStat_CheckPointer), fpin) !=
+ sizeof(PgStat_CheckPointer))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
+ MemSet(&StatsShmem->checkpointer_stats, 0, sizeof(PgStat_CheckPointer));
goto done;
}
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'T' A PgStat_StatTabEntry follows.
- */
- case 'T':
- if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
- fpin) != sizeof(PgStat_StatTabEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(tabentry, &tabbuf, sizeof(tabbuf));
- break;
-
- /*
- * 'F' A PgStat_StatFuncEntry follows.
- */
- case 'F':
- if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
- fpin) != sizeof(PgStat_StatFuncEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(funcentry, &funcbuf, sizeof(funcbuf));
- break;
-
- /*
- * 'E' The EOF marker of a complete stats file.
- */
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts. The caller must
- * rely on timestamp stored in *ts iff the function returns true.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- PgStat_WalStats myWalStats;
- PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
- PgStat_ReplSlotStats myReplSlotStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
/*
* Read archiver stats struct
*/
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
+ if (fread(&StatsShmem->archiver_stats, 1, sizeof(PgStat_Archiver),
+ fpin) != sizeof(PgStat_Archiver))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
+ MemSet(&StatsShmem->archiver_stats, 0, sizeof(PgStat_Archiver));
+ goto done;
}
/*
* Read WAL stats struct
*/
- if (fread(&myWalStats, 1, sizeof(myWalStats), fpin) != sizeof(myWalStats))
+ if (fread(&StatsShmem->wal_stats, 1, sizeof(PgStat_Wal), fpin)
+ != sizeof(PgStat_Wal))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
+ MemSet(&StatsShmem->wal_stats, 0, sizeof(PgStat_Wal));
+ goto done;
}
/*
* Read SLRU stats struct
*/
- if (fread(mySLRUStats, 1, sizeof(mySLRUStats), fpin) != sizeof(mySLRUStats))
+ if (fread(&StatsShmem->slru_stats, 1, sizeof(PgStatSharedSLRUStats),
+ fpin) != sizeof(PgStatSharedSLRUStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- /*
- * 'R' A PgStat_ReplSlotStats struct describing a replication
- * slot follows.
- */
- case 'R':
- if (fread(&myReplSlotStats, 1, sizeof(PgStat_ReplSlotStats), fpin)
- != sizeof(PgStat_ReplSlotStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
- break;
-
- case 'E':
- goto done;
-
- default:
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
- }
+ goto done;
}
-done:
- FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
/*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
+ while ((tag = fgetc(fpin)) == 'S')
{
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
+ PgStatHashKey key;
+ PgStat_StatEntryHeader *p;
+ size_t len;
CHECK_FOR_INTERRUPTS();
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
+ if (fread(&key, 1, sizeof(key), fpin) != sizeof(key))
{
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
}
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- ereport(LOG,
- (errmsg("statistics collector's time %s is later than backend local time %s",
- filetime, mytime)));
- pfree(filetime);
- pfree(mytime);
- }
+ p = get_stat_entry(key.type, key.databaseid, key.objectid,
+ false, true, &found);
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
+ /* don't allow duplicate entries */
+ if (found)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
}
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
+ /* Avoid overwriting header part */
+ len = PGSTAT_SHENT_BODY_LEN(key.type);
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
+ if (fread(PGSTAT_SHENT_BODY(p), 1, len, fpin) != len)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
}
- if (count >= PGSTAT_POLL_LOOP_COUNT)
+ if (tag != 'E')
+ {
ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
+done:
+ FreeFile(fpin);
+
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
+
+ return;
}
-
/* ----------
* pgstat_setup_memcxt() -
*
- * Create pgStatLocalContext, if not already done.
+ * Create pgStatLocalContext if not already done.
* ----------
*/
static void
pgstat_setup_memcxt(void)
{
if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
-}
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+ if (!pgStatCacheContext)
+ pgStatCacheContext =
+ AllocSetContextCreate(CacheMemoryContext,
+ "Activity statistics",
+ ALLOCSET_SMALL_SIZES);
+}
/* ----------
* pgstat_clear_snapshot() -
@@ -6180,906 +5877,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- ereport(LOG,
- (errmsg("stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid)));
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum = tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- tabentry->inserts_since_vacuum = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum += tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_WAL)
- {
- /* Reset the WAL statistics for the cluster. */
- memset(&walStats, 0, sizeof(walStats));
- walStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_resetslrucounter() -
- *
- * Reset some SLRU statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len)
-{
- int i;
- TimestampTz ts = GetCurrentTimestamp();
-
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /* reset entry with the given index, or all entries (index is -1) */
- if ((msg->m_index == -1) || (msg->m_index == i))
- {
- memset(&slruStats[i], 0, sizeof(slruStats[i]));
- slruStats[i].stat_reset_timestamp = ts;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_resetreplslotcounter() -
- *
- * Reset some replication slot statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
- int len)
-{
- int i;
- int idx = -1;
- TimestampTz ts;
-
- ts = GetCurrentTimestamp();
- if (msg->clearall)
- {
- for (i = 0; i < nReplSlotStats; i++)
- pgstat_reset_replslot(i, ts);
- }
- else
- {
- /* Get the index of replication slot statistics to reset */
- idx = pgstat_replslot_index(msg->m_slotname, false);
-
- /*
- * Nothing to do if the given slot entry is not found. This could
- * happen when the slot with the given name is removed and the
- * corresponding statistics entry is also removed before receiving the
- * reset message.
- */
- if (idx < 0)
- return;
-
- /* Reset the stats for the requested replication slot */
- pgstat_reset_replslot(idx, ts);
- }
-}
-
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signaling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * It is quite possible that a non-aggressive VACUUM ended up skipping
- * various pages, however, we'll zero the insert counter here regardless.
- * It's currently used only to track when we need to perform an "insert"
- * autovacuum, which are mainly intended to freeze newly inserted tuples.
- * Zeroing this may just mean we'll not try to vacuum the table again
- * until enough tuples have been inserted to trigger another insert
- * autovacuum. An anti-wraparound autovacuum will catch any persistent
- * stragglers.
- */
- tabentry->inserts_since_vacuum = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_wal() -
- *
- * Process a WAL message.
- * ----------
- */
-static void
-pgstat_recv_wal(PgStat_MsgWal *msg, int len)
-{
- walStats.wal_records += msg->m_wal_records;
- walStats.wal_fpi += msg->m_wal_fpi;
- walStats.wal_bytes += msg->m_wal_bytes;
- walStats.wal_buffers_full += msg->m_wal_buffers_full;
-}
-
-/* ----------
- * pgstat_recv_slru() -
- *
- * Process a SLRU message.
- * ----------
- */
-static void
-pgstat_recv_slru(PgStat_MsgSLRU *msg, int len)
-{
- slruStats[msg->m_index].blocks_zeroed += msg->m_blocks_zeroed;
- slruStats[msg->m_index].blocks_hit += msg->m_blocks_hit;
- slruStats[msg->m_index].blocks_read += msg->m_blocks_read;
- slruStats[msg->m_index].blocks_written += msg->m_blocks_written;
- slruStats[msg->m_index].blocks_exists += msg->m_blocks_exists;
- slruStats[msg->m_index].flush += msg->m_flush;
- slruStats[msg->m_index].truncate += msg->m_truncate;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_replslot() -
- *
- * Process a REPLSLOT message.
- * ----------
- */
-static void
-pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len)
-{
- int idx;
-
- /*
- * Get the index of replication slot statistics. On dropping, we don't
- * create the new statistics.
- */
- idx = pgstat_replslot_index(msg->m_slotname, !msg->m_drop);
-
- /*
- * The slot entry is not found or there is no space to accommodate the new
- * entry. This could happen when the message for the creation of a slot
- * reached before the drop message even though the actual operations
- * happen in reverse order. In such a case, the next update of the
- * statistics for the same slot will create the required entry.
- */
- if (idx < 0)
- return;
-
- /* it must be a valid replication slot index */
- Assert(idx < nReplSlotStats);
-
- if (msg->m_drop)
- {
- /* Remove the replication slot statistics with the given name */
- if (idx < nReplSlotStats - 1)
- memcpy(&replSlotStats[idx], &replSlotStats[nReplSlotStats - 1],
- sizeof(PgStat_ReplSlotStats));
- nReplSlotStats--;
- }
- else
- {
- /* Update the replication slot statistics */
- replSlotStats[idx].spill_txns += msg->m_spill_txns;
- replSlotStats[idx].spill_count += msg->m_spill_count;
- replSlotStats[idx].spill_bytes += msg->m_spill_bytes;
- replSlotStats[idx].stream_txns += msg->m_stream_txns;
- replSlotStats[idx].stream_count += msg->m_stream_count;
- replSlotStats[idx].stream_bytes += msg->m_stream_bytes;
- }
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
+ }
+
+ /* Invalidate the simple cache keys */
+ cached_dbent_key = stathashkey_zero;
+ cached_tabent_key = stathashkey_zero;
+ cached_funcent_key = stathashkey_zero;
+ cached_archiverstats_is_valid = false;
+ cached_bgwriterstats_is_valid = false;
+ cached_checkpointerstats_is_valid = false;
+ cached_walstats_is_valid = false;
+ cached_slrustats_is_valid = false;
+ n_cached_replslotstats = -1;
}
/*
@@ -7126,60 +5942,6 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
-/* ----------
- * pgstat_replslot_index
- *
- * Return the index of entry of a replication slot with the given name, or
- * -1 if the slot is not found.
- *
- * create_it tells whether to create the new slot entry if it is not found.
- * ----------
- */
-static int
-pgstat_replslot_index(const char *name, bool create_it)
-{
- int i;
-
- Assert(nReplSlotStats <= max_replication_slots);
- for (i = 0; i < nReplSlotStats; i++)
- {
- if (strcmp(replSlotStats[i].slotname, name) == 0)
- return i; /* found */
- }
-
- /*
- * The slot is not found. We don't want to register the new statistics if
- * the list is already full or the caller didn't request.
- */
- if (i == max_replication_slots || !create_it)
- return -1;
-
- /* Register new slot */
- memset(&replSlotStats[nReplSlotStats], 0, sizeof(PgStat_ReplSlotStats));
- strlcpy(replSlotStats[nReplSlotStats].slotname, name, NAMEDATALEN);
-
- return nReplSlotStats++;
-}
-
-/* ----------
- * pgstat_reset_replslot
- *
- * Reset the replication slot stats at index 'i'.
- * ----------
- */
-static void
-pgstat_reset_replslot(int i, TimestampTz ts)
-{
- /* reset only counters. Don't clear slot name */
- replSlotStats[i].spill_txns = 0;
- replSlotStats[i].spill_count = 0;
- replSlotStats[i].spill_bytes = 0;
- replSlotStats[i].stream_txns = 0;
- replSlotStats[i].stream_count = 0;
- replSlotStats[i].stream_bytes = 0;
- replSlotStats[i].stat_reset_timestamp = ts;
-}
-
/*
* pgstat_slru_index
*
@@ -7224,7 +5986,7 @@ pgstat_slru_name(int slru_idx)
* Returns pointer to entry with counters for given SLRU (based on the name
* stored in SlruCtl as lwlock tranche name).
*/
-static inline PgStat_MsgSLRU *
+static PgStat_SLRUStats *
slru_entry(int slru_idx)
{
/*
@@ -7235,7 +5997,7 @@ slru_entry(int slru_idx)
Assert((slru_idx >= 0) && (slru_idx < SLRU_NUM_ELEMENTS));
- return &SLRUStats[slru_idx];
+ return &local_SLRUStats[slru_idx];
}
/*
@@ -7245,41 +6007,48 @@ slru_entry(int slru_idx)
void
pgstat_count_slru_page_zeroed(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_zeroed += 1;
+ slru_entry(slru_idx)->blocks_zeroed += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_hit(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_hit += 1;
+ slru_entry(slru_idx)->blocks_hit += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_exists(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_exists += 1;
+ slru_entry(slru_idx)->blocks_exists += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_read(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_read += 1;
+ slru_entry(slru_idx)->blocks_read += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_written(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_written += 1;
+ slru_entry(slru_idx)->blocks_written += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_flush(int slru_idx)
{
- slru_entry(slru_idx)->m_flush += 1;
+ slru_entry(slru_idx)->flush += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_truncate(int slru_idx)
{
- slru_entry(slru_idx)->m_truncate += 1;
+ slru_entry(slru_idx)->truncate += 1;
+ have_slrustats = true;
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index af91c313e2..9b9c9b1c11 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -251,7 +251,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -512,7 +511,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1327,12 +1325,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1782,11 +1774,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2699,8 +2686,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3029,8 +3014,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3097,13 +3080,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3176,22 +3152,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3653,22 +3613,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3895,8 +3839,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3920,8 +3862,7 @@ PostmasterStateMachine(void)
{
/*
* PM_WAIT_DEAD_END state ends when the BackendList is entirely empty
- * (ie, no dead_end children remain), and the archiver and stats
- * collector are gone too.
+ * (ie, no dead_end children remain), and the archiveris gone too.
*
* The reason we wait for those two is to protect them against a new
* postmaster starting conflicting subprocesses; this isn't an
@@ -3931,8 +3872,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4133,8 +4073,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5067,18 +5005,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5191,12 +5117,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6113,7 +6033,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6169,8 +6088,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6403,7 +6320,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 0f54635550..d21801cf90 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1568,8 +1568,8 @@ is_checksummed_file(const char *fullpath, const char *filename)
*
* If 'missing_ok' is true, will not throw an error if the file is not found.
*
- * If dboid is anything other than InvalidOid then any checksum failures detected
- * will get reported to the stats collector.
+ * If dboid is anything other than InvalidOid then any checksum failures
+ * detected will get reported to the activity stats facility.
*
* Returns true if the file was successfully sent, false if 'missing_ok',
* and the file did not exist.
diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c
index e00c7ffc01..608818beea 100644
--- a/src/backend/replication/slot.c
+++ b/src/backend/replication/slot.c
@@ -692,14 +692,10 @@ ReplicationSlotDropPtr(ReplicationSlot *slot)
(errmsg("could not remove directory \"%s\"", tmppath)));
/*
- * Send a message to drop the replication slot to the stats collector.
- * Since there is no guarantee of the order of message transfer on a UDP
- * connection, it's possible that a message for creating a new slot
- * reaches before a message for removing the old slot. We send the drop
- * and create messages while holding ReplicationSlotAllocationLock to
- * reduce that possibility. If the messages reached in reverse, we would
- * lose one statistics update message. But the next update message will
- * create the statistics for the replication slot.
+ * Drop the statistics entry for the replication slot. Do this while
+ * holding ReplicationSlotAllocationLock so that we don't drop a statistics
+ * entry for another slot with the same name just created on another
+ * session.
*/
if (SlotIsLogical(slot))
pgstat_report_replslot_drop(NameStr(slot->data.name));
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 71b5852224..160977d50d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2047,7 +2047,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ CheckPointerStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2157,7 +2157,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2347,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2355,7 +2355,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index f9bbe97b50..78cfe91eab 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -149,6 +149,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -267,6 +268,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index db7e59f8b7..e6e4c0fb04 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -177,7 +177,9 @@ static const char *const BuiltinTrancheNames[] = {
/* LWTRANCHE_PARALLEL_APPEND: */
"ParallelAppend",
/* LWTRANCHE_PER_XACT_PREDICATE_LIST: */
- "PerXactPredicateList"
+ "PerXactPredicateList",
+ /* LWTRANCHE_STATS: */
+ "ActivityStatistics"
};
StaticAssertDecl(lengthof(BuiltinTrancheNames) ==
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index 774292fd94..91bf8d5b5d 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -53,3 +53,4 @@ XactTruncationLock 44
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+StatsLock 48
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 0f31ff3822..fc96f21519 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -414,8 +414,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
DropRelFileNodesAllBuffers(rnodes, nrels);
/*
- * It'd be nice to tell the stats collector to forget them immediately,
- * too. But we can't because we don't know the OIDs.
+ * It'd be nice to tell the activity stats facility to forget them
+ * immediately, too. But we can't because we don't know the OIDs.
*/
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 28055680aa..8b51852505 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3271,6 +3271,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3842,6 +3848,7 @@ PostgresMain(int argc, char *argv[],
volatile bool send_ready_for_query = true;
bool idle_in_transaction_timeout_enabled = false;
bool idle_session_timeout_enabled = false;
+ bool idle_stats_update_timeout_enabled = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4238,11 +4245,12 @@ PostgresMain(int argc, char *argv[],
* Note: this includes fflush()'ing the last of the prior output.
*
* This is also a good time to send collected statistics to the
- * collector, and to update the PS stats display. We avoid doing
- * those every time through the message loop because it'd slow down
- * processing of batched messages, and because we don't want to report
- * uncommitted updates (that confuses autovacuum). The notification
- * processor wants a call too, if we are not in a transaction block.
+ * activity statistics, and to update the PS stats display. We avoid
+ * doing those every time through the message loop because it'd slow
+ * down processing of batched messages, and because we don't want to
+ * report uncommitted updates (that confuses autovacuum). The
+ * notification processor wants a call too, if we are not in a
+ * transaction block.
*
* Also, if an idle timeout is enabled, start the timer for that.
*/
@@ -4276,6 +4284,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4288,8 +4298,14 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ /* Start the idle-stats-update timer */
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ idle_stats_update_timeout_enabled = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
@@ -4323,9 +4339,9 @@ PostgresMain(int argc, char *argv[],
firstchar = ReadCommand(&input_message);
/*
- * (4) turn off the idle-in-transaction and idle-session timeouts, if
- * active. We do this before step (5) so that any last-moment timeout
- * is certain to be detected in step (5).
+ * (4) turn off the idle-in-transaction, idle-session and
+ * idle-state-update timeouts if active. We do this before step (5) so
+ * that any last-moment timeout is certain to be detected in step (5).
*
* At most one of these timeouts will be active, so there's no need to
* worry about combining the timeout.c calls into one.
@@ -4340,6 +4356,11 @@ PostgresMain(int argc, char *argv[],
disable_timeout(IDLE_SESSION_TIMEOUT, false);
idle_session_timeout_enabled = false;
}
+ if (idle_stats_update_timeout_enabled)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ idle_stats_update_timeout_enabled = false;
+ }
/*
* (5) disable async signal conditions again.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5c12a165a1..fd6465dfb9 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1,7 +1,7 @@
/*-------------------------------------------------------------------------
*
* pgstatfuncs.c
- * Functions for accessing the statistics collector data
+ * Functions for accessing the activity statistics data
*
* Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -35,9 +35,6 @@
#define HAS_PGSTAT_PERMISSIONS(role) (is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS) || has_privs_of_role(GetUserId(), role))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1269,7 +1266,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1285,7 +1282,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1301,7 +1298,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1317,7 +1314,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1333,7 +1330,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1349,7 +1346,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1365,7 +1362,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1381,7 +1378,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1397,7 +1394,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1430,7 +1427,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1446,7 +1443,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1461,7 +1458,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1476,7 +1473,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1491,7 +1488,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1506,7 +1503,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1521,7 +1518,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1536,11 +1533,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1555,7 +1552,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1573,7 +1570,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1610,7 +1607,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1626,7 +1623,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1634,69 +1631,71 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
Datum
pg_stat_get_bgwriter_timed_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->timed_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->timed_checkpoints);
}
Datum
pg_stat_get_bgwriter_requested_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->requested_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->requested_checkpoints);
}
Datum
pg_stat_get_bgwriter_buf_written_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_checkpoints);
}
Datum
pg_stat_get_bgwriter_buf_written_clean(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_clean);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_written_clean);
}
Datum
pg_stat_get_bgwriter_maxwritten_clean(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->maxwritten_clean);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->maxwritten_clean);
}
Datum
pg_stat_get_checkpoint_write_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
- PG_RETURN_FLOAT8((double) pgstat_fetch_global()->checkpoint_write_time);
+ PG_RETURN_FLOAT8((double)
+ pgstat_fetch_stat_checkpointer()->checkpoint_write_time);
}
Datum
pg_stat_get_checkpoint_sync_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
- PG_RETURN_FLOAT8((double) pgstat_fetch_global()->checkpoint_sync_time);
+ PG_RETURN_FLOAT8((double)
+ pgstat_fetch_stat_checkpointer()->checkpoint_sync_time);
}
Datum
pg_stat_get_bgwriter_stat_reset_time(PG_FUNCTION_ARGS)
{
- PG_RETURN_TIMESTAMPTZ(pgstat_fetch_global()->stat_reset_timestamp);
+ PG_RETURN_TIMESTAMPTZ(pgstat_fetch_stat_bgwriter()->stat_reset_timestamp);
}
Datum
pg_stat_get_buf_written_backend(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_backend);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_backend);
}
Datum
pg_stat_get_buf_fsync_backend(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_fsync_backend);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_fsync_backend);
}
Datum
pg_stat_get_buf_alloc(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_alloc);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_alloc);
}
/*
@@ -1710,7 +1709,7 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_WAL_COLS];
bool nulls[PG_STAT_GET_WAL_COLS];
char buf[256];
- PgStat_WalStats *wal_stats;
+ PgStat_Wal *wal_stats;
/* Initialise values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
@@ -1735,11 +1734,11 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
wal_stats = pgstat_fetch_stat_wal();
/* Fill values and NULLs */
- values[0] = Int64GetDatum(wal_stats->wal_records);
- values[1] = Int64GetDatum(wal_stats->wal_fpi);
+ values[0] = Int64GetDatum(wal_stats->wal_usage.wal_records);
+ values[1] = Int64GetDatum(wal_stats->wal_usage.wal_fpi);
/* Convert to numeric. */
- snprintf(buf, sizeof buf, UINT64_FORMAT, wal_stats->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, wal_stats->wal_usage.wal_bytes);
values[2] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
@@ -2020,7 +2019,7 @@ pg_stat_get_xact_function_self_time(PG_FUNCTION_ARGS)
Datum
pg_stat_get_snapshot_timestamp(PG_FUNCTION_ARGS)
{
- PG_RETURN_TIMESTAMPTZ(pgstat_fetch_global()->stats_timestamp);
+ PG_RETURN_TIMESTAMPTZ(pgstat_get_stat_timestamp());
}
/* Discard the active statistics snapshot */
@@ -2108,7 +2107,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDesc tupdesc;
Datum values[7];
bool nulls[7];
- PgStat_ArchiverStats *archiver_stats;
+ PgStat_Archiver *archiver_stats;
/* Initialise values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
@@ -2178,7 +2177,7 @@ pg_stat_get_replication_slots(PG_FUNCTION_ARGS)
Tuplestorestate *tupstore;
MemoryContext per_query_ctx;
MemoryContext oldcontext;
- PgStat_ReplSlotStats *slotstats;
+ PgStat_ReplSlot *slotstats;
int nstats;
int i;
@@ -2211,7 +2210,7 @@ pg_stat_get_replication_slots(PG_FUNCTION_ARGS)
{
Datum values[PG_STAT_GET_REPLICATION_SLOT_COLS];
bool nulls[PG_STAT_GET_REPLICATION_SLOT_COLS];
- PgStat_ReplSlotStats *s = &(slotstats[i]);
+ PgStat_ReplSlot *s = &(slotstats[i]);
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..0762c2034c 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -72,6 +72,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
+#include "pgstat.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rowsecurity.h"
#include "storage/lmgr.h"
@@ -2366,6 +2367,10 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
*/
RelationCloseSmgr(relation);
+ /* break mutual link with stats entry */
+ pgstat_delinkstats(relation);
+
+ if (relation->rd_rel)
/*
* Free all the subsidiary data structures of the relcache entry, then the
* entry itself.
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index ea28769d6a..997afcab6d 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -34,6 +34,7 @@ volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 0f67b99cc5..2567668b6c 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -269,9 +269,6 @@ GetBackendTypeDesc(BackendType backendType)
case B_ARCHIVER:
backendDesc = "archiver";
break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
case B_LOGGER:
backendDesc = "logger";
break;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e5965bc517..d4c17fd7ab 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
static void IdleSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -621,6 +622,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
RegisterTimeout(IDLE_SESSION_TIMEOUT, IdleSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1243,6 +1246,14 @@ IdleSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 17579eeaca..85299e2138 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -746,8 +746,8 @@ const char *const config_group_names[] =
gettext_noop("Statistics"),
/* STATS_MONITORING */
gettext_noop("Statistics / Monitoring"),
- /* STATS_COLLECTOR */
- gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* STATS_ACTIVITY */
+ gettext_noop("Statistics / Query and Index Activity Statistics"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
@@ -1457,7 +1457,7 @@ static struct config_bool ConfigureNamesBool[] =
#endif
{
- {"track_activities", PGC_SUSET, STATS_COLLECTOR,
+ {"track_activities", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects information about executing commands."),
gettext_noop("Enables the collection of information on the currently "
"executing command of each session, along with "
@@ -1468,7 +1468,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_counts", PGC_SUSET, STATS_COLLECTOR,
+ {"track_counts", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects statistics on database activity."),
NULL
},
@@ -1477,7 +1477,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_io_timing", PGC_SUSET, STATS_COLLECTOR,
+ {"track_io_timing", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects timing statistics for database I/O activity."),
NULL
},
@@ -4356,7 +4356,7 @@ static struct config_string ConfigureNamesString[] =
},
{
- {"stats_temp_directory", PGC_SIGHUP, STATS_COLLECTOR,
+ {"stats_temp_directory", PGC_SIGHUP, STATS_ACTIVITY,
gettext_noop("Writes temporary statistics files to the specified directory."),
NULL,
GUC_SUPERUSER_ONLY
@@ -4692,7 +4692,7 @@ static struct config_enum ConfigureNamesEnum[] =
},
{
- {"track_functions", PGC_SUSET, STATS_COLLECTOR,
+ {"track_functions", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects function-level statistics on database activity."),
NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 8930a94fff..4f5b6bdb12 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -580,7 +580,7 @@
# STATISTICS
#------------------------------------------------------------------------------
-# - Query and Index Statistics Collector -
+# - Query and Index Activity Statistics -
#track_activities = on
#track_counts = on
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index f674a7c94e..c61b828bf3 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 109;
+use Test::More tests => 108;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -124,7 +124,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index adb9f819bb..12708b9470 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -84,6 +84,8 @@ extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -322,7 +324,6 @@ typedef enum BackendType
B_WAL_SENDER,
B_WAL_WRITER,
B_ARCHIVER,
- B_STATS_COLLECTOR,
B_LOGGER,
} BackendType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c38b689710..0472b728bf 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2021, PostgreSQL Global Development Group
*
@@ -12,12 +12,15 @@
#define PGSTAT_H
#include "datatype/timestamp.h"
+#include "executor/instrument.h"
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -27,8 +30,8 @@
* ----------
*/
#define PGSTAT_STAT_PERMANENT_DIRECTORY "pg_stat"
-#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
-#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
+#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/saved_stats"
+#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/saved_stats.tmp"
/* Default directory to store temporary statistics data in */
#define PG_STAT_TMP_DIR "pg_stat_tmp"
@@ -41,38 +44,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_RESETSLRUCOUNTER,
- PGSTAT_MTYPE_RESETREPLSLOTCOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_WAL,
- PGSTAT_MTYPE_SLRU,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE,
- PGSTAT_MTYPE_REPLSLOT,
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -83,9 +54,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -159,10 +129,13 @@ typedef enum PgStat_Single_Reset_Type
*/
typedef struct PgStat_TableStatus
{
- Oid t_id; /* table's OID */
- bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
+ Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
/* ----------
@@ -186,353 +159,57 @@ typedef struct PgStat_TableXactStatus
struct PgStat_TableXactStatus *next; /* next of same subxact */
} PgStat_TableXactStatus;
-
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
+/*
+ * Archiver statistics kept in the shared stats
*/
-typedef struct PgStat_MsgHdr
+typedef struct PgStat_Archiver
{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
+ PgStat_Counter archived_count; /* archival successes */
+ char last_archived_wal[MAX_XFN_CHARS + 1]; /* last WAL file
+ * archived */
+ TimestampTz last_archived_timestamp; /* last archival success time */
+ PgStat_Counter failed_count; /* failed archival attempts */
+ char last_failed_wal[MAX_XFN_CHARS + 1]; /* WAL file involved in
+ * last failure */
+ TimestampTz last_failed_timestamp; /* last archival failure time */
+ TimestampTz stat_reset_timestamp;
+} PgStat_Archiver;
/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgDummy
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_alloc;
+ TimestampTz stat_reset_timestamp;
+} PgStat_BgWriter;
/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
+ * PgStat_CheckPointer checkpointer statistics
* ----------
*/
-
-typedef struct PgStat_MsgInquiry
+typedef struct PgStat_CheckPointer
{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgResetslrucounter Sent by the backend to tell the collector
- * to reset a SLRU counter
- * ----------
- */
-typedef struct PgStat_MsgResetslrucounter
-{
- PgStat_MsgHdr m_hdr;
- int m_index;
-} PgStat_MsgResetslrucounter;
-
-/* ----------
- * PgStat_MsgResetreplslotcounter Sent by the backend to tell the collector
- * to reset replication slot counter(s)
- * ----------
- */
-typedef struct PgStat_MsgResetreplslotcounter
-{
- PgStat_MsgHdr m_hdr;
- char m_slotname[NAMEDATALEN];
- bool clearall;
-} PgStat_MsgResetreplslotcounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgWal Sent by backends and background processes to update WAL statistics.
- * ----------
- */
-typedef struct PgStat_MsgWal
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_wal_records;
- PgStat_Counter m_wal_fpi;
- uint64 m_wal_bytes;
- PgStat_Counter m_wal_buffers_full;
-} PgStat_MsgWal;
-
-/* ----------
- * PgStat_MsgSLRU Sent by a backend to update SLRU statistics.
- * ----------
- */
-typedef struct PgStat_MsgSLRU
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_index;
- PgStat_Counter m_blocks_zeroed;
- PgStat_Counter m_blocks_hit;
- PgStat_Counter m_blocks_read;
- PgStat_Counter m_blocks_written;
- PgStat_Counter m_blocks_exists;
- PgStat_Counter m_flush;
- PgStat_Counter m_truncate;
-} PgStat_MsgSLRU;
-
-/* ----------
- * PgStat_MsgReplSlot Sent by a backend or a wal sender to update replication
- * slot statistics.
- * ----------
- */
-typedef struct PgStat_MsgReplSlot
-{
- PgStat_MsgHdr m_hdr;
- char m_slotname[NAMEDATALEN];
- bool m_drop;
- PgStat_Counter m_spill_txns;
- PgStat_Counter m_spill_count;
- PgStat_Counter m_spill_bytes;
- PgStat_Counter m_stream_txns;
- PgStat_Counter m_stream_count;
- PgStat_Counter m_stream_bytes;
-} PgStat_MsgReplSlot;
-
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_CheckPointer;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -548,7 +225,6 @@ typedef struct PgStat_FunctionCounts
*/
typedef struct PgStat_BackendFunctionEntry
{
- Oid f_id;
PgStat_FunctionCounts f_counts;
} PgStat_BackendFunctionEntry;
@@ -564,101 +240,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgResetslrucounter msg_resetslrucounter;
- PgStat_MsgResetreplslotcounter msg_resetreplslotcounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgWal msg_wal;
- PgStat_MsgSLRU msg_slru;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
- PgStat_MsgReplSlot msg_replslot;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -667,13 +250,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9F
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -683,7 +262,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -693,29 +271,86 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatEntryHead common header struct for PgStat_Stat*Entry
+ * ----------
+ */
+typedef struct PgStat_StatEntryHeader
+{
+ LWLock lock;
+ bool dropped; /* This entry is being dropped and should
+ * be removed when refcount goes to
+ * zero. */
+ pg_atomic_uint32 refcount; /* How many backends are referenceing */
+} PgStat_StatEntryHeader;
+
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/* ----------
+ * PgStat_Wal Sent by backends and background processes to
+ * update WAL statistics.
+ * ----------
+ */
+typedef struct PgStat_Wal
+{
+ WalUsage wal_usage;
+ PgStat_Counter wal_buffers_full;
+ TimestampTz stat_reset_timestamp;
+} PgStat_Wal;
+
+/*
+ * SLRU statistics kept in the stats collector
+ */
+typedef struct PgStat_SLRUStats
+{
+ PgStat_Counter blocks_zeroed;
+ PgStat_Counter blocks_hit;
+ PgStat_Counter blocks_read;
+ PgStat_Counter blocks_written;
+ PgStat_Counter blocks_exists;
+ PgStat_Counter flush;
+ PgStat_Counter truncate;
+ TimestampTz stat_reset_timestamp;
+} PgStat_SLRUStats;
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
- Oid tableid;
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ /* Persistent data follow */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -735,99 +370,35 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
{
- Oid functionid;
-
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ /* Persistent data follow */
PgStat_Counter f_numcalls;
PgStat_Counter f_total_time; /* times in microseconds */
PgStat_Counter f_self_time;
} PgStat_StatFuncEntry;
-
-/*
- * Archiver statistics kept in the stats collector
- */
-typedef struct PgStat_ArchiverStats
-{
- PgStat_Counter archived_count; /* archival successes */
- char last_archived_wal[MAX_XFN_CHARS + 1]; /* last WAL file
- * archived */
- TimestampTz last_archived_timestamp; /* last archival success time */
- PgStat_Counter failed_count; /* failed archival attempts */
- char last_failed_wal[MAX_XFN_CHARS + 1]; /* WAL file involved in
- * last failure */
- TimestampTz last_failed_timestamp; /* last archival failure time */
- TimestampTz stat_reset_timestamp;
-} PgStat_ArchiverStats;
-
-/*
- * Global statistics kept in the stats collector
- */
-typedef struct PgStat_GlobalStats
-{
- TimestampTz stats_timestamp; /* time of stats file update */
- PgStat_Counter timed_checkpoints;
- PgStat_Counter requested_checkpoints;
- PgStat_Counter checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter checkpoint_sync_time;
- PgStat_Counter buf_written_checkpoints;
- PgStat_Counter buf_written_clean;
- PgStat_Counter maxwritten_clean;
- PgStat_Counter buf_written_backend;
- PgStat_Counter buf_fsync_backend;
- PgStat_Counter buf_alloc;
- TimestampTz stat_reset_timestamp;
-} PgStat_GlobalStats;
-
-/*
- * WAL statistics kept in the stats collector
- */
-typedef struct PgStat_WalStats
-{
- PgStat_Counter wal_records;
- PgStat_Counter wal_fpi;
- uint64 wal_bytes;
- PgStat_Counter wal_buffers_full;
- TimestampTz stat_reset_timestamp;
-} PgStat_WalStats;
-
-/*
- * SLRU statistics kept in the stats collector
- */
-typedef struct PgStat_SLRUStats
-{
- PgStat_Counter blocks_zeroed;
- PgStat_Counter blocks_hit;
- PgStat_Counter blocks_read;
- PgStat_Counter blocks_written;
- PgStat_Counter blocks_exists;
- PgStat_Counter flush;
- PgStat_Counter truncate;
- TimestampTz stat_reset_timestamp;
-} PgStat_SLRUStats;
-
/*
* Replication slot statistics kept in the stats collector
*/
-typedef struct PgStat_ReplSlotStats
+typedef struct PgStat_ReplSlot
{
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
char slotname[NAMEDATALEN];
PgStat_Counter spill_txns;
PgStat_Counter spill_count;
@@ -836,7 +407,7 @@ typedef struct PgStat_ReplSlotStats
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
TimestampTz stat_reset_timestamp;
-} PgStat_ReplSlotStats;
+} PgStat_ReplSlot;
/* ----------
* Backend states
@@ -885,7 +456,7 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
+ WAIT_EVENT_READING_STATS_FILE,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1138,7 +709,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1335,18 +906,26 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
+
+/*
+ * CheckPointer statistics counters are updated directly by checkpointer and
+ * bufmgr
+ */
+extern PgStat_CheckPointer CheckPointerStats;
/*
* WAL statistics counter is updated by backends and background processes
*/
-extern PgStat_MsgWal WalStats;
+extern PgStat_Wal WalStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1361,33 +940,27 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
extern void pgstat_reset_replslot_counter(const char *name);
+extern void pgstat_report_wal(void);
extern void pgstat_report_autovac(Oid dboid);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples);
@@ -1427,6 +1000,7 @@ extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
extern void pgstat_initstats(Relation rel);
+extern void pgstat_delinkstats(Relation rel);
extern char *pgstat_clip_activity(const char *raw_activity);
@@ -1549,9 +1123,10 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
-extern void pgstat_send_wal(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
+extern void pgstat_report_checkpointer(void);
+extern void pgstat_report_wal(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1560,15 +1135,20 @@ extern void pgstat_send_wal(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_extended(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
-extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
-extern PgStat_GlobalStats *pgstat_fetch_global(void);
-extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
+extern TimestampTz pgstat_get_stat_timestamp(void);
+extern PgStat_Archiver *pgstat_fetch_stat_archiver(void);
+extern PgStat_BgWriter *pgstat_fetch_stat_bgwriter(void);
+extern PgStat_CheckPointer *pgstat_fetch_stat_checkpointer(void);
+extern PgStat_Wal *pgstat_fetch_stat_wal(void);
extern PgStat_SLRUStats *pgstat_fetch_slru(void);
-extern PgStat_ReplSlotStats *pgstat_fetch_replslot(int *nslots_p);
+extern PgStat_ReplSlot *pgstat_fetch_replslot(int *nslots_p);
extern void pgstat_count_slru_page_zeroed(int slru_idx);
extern void pgstat_count_slru_page_hit(int slru_idx);
@@ -1579,5 +1159,6 @@ extern void pgstat_count_slru_flush(int slru_idx);
extern void pgstat_count_slru_truncate(int slru_idx);
extern const char *pgstat_slru_name(int slru_idx);
extern int pgstat_slru_index(const char *name);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index cbf2510fbf..9ed6b54428 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -218,6 +218,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_SHARED_TIDBITMAP,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_PER_XACT_PREDICATE_LIST,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index b9b5c1adda..add9c53ee3 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -88,7 +88,7 @@ enum config_group
PROCESS_TITLE,
STATS,
STATS_MONITORING,
- STATS_COLLECTOR,
+ STATS_ACTIVITY,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index ecb2a366a5..f090f7372a 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -32,6 +32,7 @@ typedef enum TimeoutId
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IDLE_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.27.0
* [PATCH v46 4/7] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 1 second and such large data
is serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 10 seconds.
Until 60 second has elapsed since the last flushing to shared stats,
lock failure postpones stats flushing to try to alleviate lock
contention that slows down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/heap/heapam_handler.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/access/transam/xlog.c | 6 +-
src/backend/catalog/index.c | 24 +-
src/backend/commands/analyze.c | 8 +-
src/backend/commands/dbcommands.c | 2 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/vacuum.c | 4 +-
src/backend/postmaster/autovacuum.c | 76 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 26 +-
src/backend/postmaster/pgarch.c | 12 +-
src/backend/postmaster/pgstat.c | 6135 +++++++----------
src/backend/postmaster/postmaster.c | 88 +-
src/backend/replication/basebackup.c | 4 +-
src/backend/replication/slot.c | 12 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 4 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/smgr.c | 4 +-
src/backend/tcop/postgres.c | 41 +-
src/backend/utils/adt/pgstatfuncs.c | 95 +-
src/backend/utils/cache/relcache.c | 5 +
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 3 -
src/backend/utils/init/postinit.c | 11 +
src/backend/utils/misc/guc.c | 14 +-
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 3 +-
src/include/pgstat.h | 727 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/guc_tables.h | 2 +-
src/include/utils/timeout.h | 1 +
35 files changed, 2791 insertions(+), 4553 deletions(-)
inline patch 1: src/backend/access/heap/heapam_handler.c, src/backend/access/heap/vacuumlazy.c, src/backend/access/transam/xlog.c and 32 more (+2791/-4553 lines) download inline-1.patch
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 4a70e20a14..a8c882d315 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1086,8 +1086,8 @@ heapam_scan_analyze_next_tuple(TableScanDesc scan, TransactionId OldestXmin,
* our own. In this case we should count and sample the row,
* to accommodate users who load a table and analyze it in one
* transaction. (pgstat_report_analyze has to adjust the
- * numbers we send to the stats collector to make this come
- * out right.)
+ * numbers we report to the activity stats facility to make
+ * this come out right.)
*/
if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(targtuple->t_data)))
{
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f3d2265fad..ceccb7e19e 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -598,7 +598,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
new_min_multi,
false);
- /* report results to the stats collector, too */
+ /* report results to the activity stats facility, too */
pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
Max(new_live_tuples, 0),
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index b18257c198..a4ab63823c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2208,7 +2208,7 @@ AdvanceXLInsertBuffer(XLogRecPtr upto, bool opportunistic)
WriteRqst.Flush = 0;
XLogWrite(WriteRqst, false);
LWLockRelease(WALWriteLock);
- WalStats.m_wal_buffers_full++;
+ WalStats.wal_buffers_full++;
TRACE_POSTGRESQL_WAL_BUFFER_WRITE_DIRTY_DONE();
}
/* Re-acquire WALBufMappingLock and retry */
@@ -8608,8 +8608,8 @@ LogCheckpointEnd(bool restartpoint)
CheckpointStats.ckpt_sync_end_t);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time += write_msecs;
- BgWriterStats.m_checkpoint_sync_time += sync_msecs;
+ CheckPointerStats.checkpoint_write_time += write_msecs;
+ CheckPointerStats.checkpoint_sync_time += sync_msecs;
/*
* All of the published timing statistics are accounted for. Only
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cffbc0ac38..3f55c34909 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1854,28 +1854,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be flushed by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Copy data of pg_statistic from the old index to the new one */
CopyStatistics(oldIndexId, newIndexId);
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 7295cf0215..308b4ab034 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -644,10 +644,10 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
}
/*
- * Report ANALYZE to the stats collector, too. However, if doing
- * inherited stats we shouldn't report, because the stats collector only
- * tracks per-table stats. Reset the changes_since_analyze counter only
- * if we analyzed all columns; otherwise, there is still work for
+ * Report ANALYZE to the activity stats facility, too. However, if doing
+ * inherited stats we shouldn't report, because the activity stats facility
+ * only tracks per-table stats. Reset the changes_since_analyze counter
+ * only if we analyzed all columns; otherwise, there is still work for
* auto-analyze to do.
*/
if (!inh)
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 2b159b60eb..acea4de382 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -971,7 +971,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
DropDatabaseBuffers(db_id);
/*
- * Tell the stats collector to forget it immediately, too.
+ * Tell the active stats facility to forget it immediately, too.
*/
pgstat_drop_database(db_id);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..1464b97c7f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -336,10 +336,10 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
/*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
+ * Inform activity stats facility about our activity: basically, we
+ * truncated the matview and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
*/
pgstat_count_truncate(matviewRel);
if (!stmt->skipData)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index b97d48ee01..e80ab5edd0 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -319,8 +319,8 @@ vacuum(List *relations, VacuumParams *params,
errmsg("VACUUM option DISABLE_PAGE_SKIPPING cannot be used with FULL")));
/*
- * Send info about dead objects to the statistics collector, unless we are
- * in autovacuum --- autovacuum.c does this for itself.
+ * Send info about dead objects to the activity statistics facility, unless
+ * we are in autovacuum --- autovacuum.c does this for itself.
*/
if ((params->options & VACOPT_VACUUM) && !IsAutoVacuumWorkerProcess())
pgstat_vacuum_stat();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 47e60ca561..194a02024b 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -342,9 +342,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1684,12 +1681,12 @@ AutoVacWorkerMain(int argc, char *argv[])
char dbname[NAMEDATALEN];
/*
- * Report autovac startup to the stats collector. We deliberately do
- * this before InitPostgres, so that the last_autovac_time will get
- * updated even if the connection attempt fails. This is to prevent
- * autovac from getting "stuck" repeatedly selecting an unopenable
- * database, rather than making any progress on stuff it can connect
- * to.
+ * Report autovac startup to the activity stats facility. We
+ * deliberately do this before InitPostgres, so that the
+ * last_autovac_time will get updated even if the connection attempt
+ * fails. This is to prevent autovac from getting "stuck" repeatedly
+ * selecting an unopenable database, rather than making any progress on
+ * stuff it can connect to.
*/
pgstat_report_autovac(dbid);
@@ -1961,8 +1958,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1981,17 +1976,11 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
/*
- * Clean up any dead statistics collector entries for this DB. We always
+ * Clean up any dead activity statistics entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
* nothing worth vacuuming in the database.
*/
@@ -2034,9 +2023,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2114,8 +2100,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2198,8 +2184,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2758,29 +2744,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2984,17 +2947,10 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared = NULL;
- PgStat_StatDBEntry *dbentry = NULL;
-
- if (classForm->relisshared)
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- else
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -3024,7 +2980,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
*
* For analyze, the analysis done is that the number of tuples inserted,
* deleted and updated since the last analyze exceeds a threshold calculated
- * in the same fashion as above. Note that the collector actually stores
+ * in the same fashion as above. Note that the activity statistics stores
* the number of tuples (both live and dead) that there were as of the last
* analyze. This is asymmetric to the VACUUM case.
*
@@ -3034,8 +2990,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
*
* A table whose autovacuum_enabled option is false is
* automatically skipped (unless we have to vacuum it due to freeze_max_age).
- * Thus autovacuum can be disabled for specific tables. Also, when the stats
- * collector does not have data about a table, it will be skipped.
+ * Thus autovacuum can be disabled for specific tables. Also, when the activity
+ * statistics does not have data about a table, it will be skipped.
*
* A table whose vac_base_thresh value is < 0 takes the base value from the
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 715d5195bb..679992dc89 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -244,9 +244,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Send off activity statistics to the activity stats facility
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 54a818bf61..309a9997e1 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -360,7 +360,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ CheckPointerStats.requested_checkpoints++;
}
/*
@@ -374,7 +374,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ CheckPointerStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -495,17 +495,11 @@ CheckpointerMain(void)
/* Check for archive_timeout and switch xlog files if necessary. */
CheckArchiveTimeout();
- /*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
- */
- pgstat_send_bgwriter();
+ /* Send off activity statistics to the activity stats facility. */
+ pgstat_report_checkpointer();
/* Send WAL statistics to the stats collector. */
- pgstat_send_wal();
+ pgstat_report_wal();
/*
* If any checkpoint flags have been set, redo the loop to handle the
@@ -711,9 +705,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Report interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_report_checkpointer();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1257,8 +1251,10 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ CheckPointerStats.buf_written_backend
+ += CheckpointerShmem->num_backend_writes;
+ CheckPointerStats.buf_fsync_backend
+ += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 9a2e21bf86..504443dcc0 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -372,20 +372,20 @@ pgarch_ArchiverCopyLoop(void)
pgarch_archiveDone(xlog);
/*
- * Tell the collector about the WAL file that we successfully
- * archived
+ * Tell the activity statistics facility about the WAL file
+ * that we successfully archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
else
{
/*
- * Tell the collector about the WAL file that we failed to
- * archive
+ * Tell the activity statistics facility about the WAL file
+ * that we failed to archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 3f24a33ef1..ecf9d9adcc 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (10000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (1000ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (60000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2021, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,13 +35,9 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
-#include "executor/instrument.h"
+#include "common/hashfn.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
@@ -54,20 +45,16 @@
#include "postmaster/postmaster.h"
#include "replication/slot.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
+#include "storage/condition_variable.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
#include "utils/timestamp.h"
@@ -75,35 +62,20 @@
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 10000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 1000 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 60000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -118,7 +90,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -133,17 +104,11 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
-/*
- * BgWriter and WAL global statistics counters.
- * Stored directly in a stats message structure so they can be sent
- * without needing to copy things around. We assume these init to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-PgStat_MsgWal WalStats;
-
/*
* WAL usage counters saved from pgWALUsage at the previous call to
* pgstat_send_wal(). This is used to calculate how much WAL usage
@@ -170,73 +135,246 @@ static const char *const slru_names[] = {
#define SLRU_NUM_ELEMENTS lengthof(slru_names)
+/* struct for shared SLRU stats */
+typedef struct PgStatSharedSLRUStats
+{
+ PgStat_SLRUStats entry[SLRU_NUM_ELEMENTS];
+ LWLock lock;
+ pg_atomic_uint32 changecount;
+} PgStatSharedSLRUStats;
+
+StaticAssertDecl(sizeof(TimestampTz) == sizeof(pg_atomic_uint64),
+ "size of pg_atomic_uint64 doesn't match TimestampTz");
+
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ int refcount; /* # of processes that is attaching the
+ * shared stats memory */
+ /* Global stats structs */
+ PgStat_Archiver archiver_stats;
+ pg_atomic_uint32 archiver_changecount;
+ PgStat_BgWriter bgwriter_stats;
+ pg_atomic_uint32 bgwriter_changecount;
+ PgStat_CheckPointer checkpointer_stats;
+ pg_atomic_uint32 checkpointer_changecount;
+ PgStat_Wal wal_stats;
+ LWLock wal_stats_lock;
+ PgStatSharedSLRUStats slru_stats;
+ pg_atomic_uint32 slru_changecount;
+ pg_atomic_uint64 stats_timestamp;
+
+ /* Reset offsets, protected by StatsLock */
+ PgStat_Archiver archiver_reset_offset;
+ PgStat_BgWriter bgwriter_reset_offset;
+ PgStat_CheckPointer checkpointer_reset_offset;
+
+ /* file read/write protection */
+ bool attach_holdoff;
+ ConditionVariable holdoff_cv;
+
+ pg_atomic_uint64 gc_count; /* # of entries deleted. not
+ * protected by StatsLock */
+} StatsShmemStruct;
+
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
+
+/* CheckPointer global statistics counters */
+PgStat_CheckPointer CheckPointerStats = {0};
+
+/* WAL global statistics counters */
+PgStat_Wal WalStats = {0} ;
+
/*
- * SLRU statistics counts waiting to be sent to the collector. These are
- * stored directly in stats message format so they can be sent without needing
- * to copy things around. We assume this variable inits to zeroes. Entries
- * are one-to-one with slru_names[].
+ * XXXX: always try to flush WAL stats. We don't want to manipulate another
+ * counter during XLogInsert so we don't have an effecient short cut to know
+ * whether any counter gets incremented.
*/
-static PgStat_MsgSLRU SLRUStats[SLRU_NUM_ELEMENTS];
+static inline bool
+walstats_pending(void)
+{
+ static const PgStat_Wal all_zeroes;
+
+ return memcmp(&WalStats, &all_zeroes,
+ offsetof(PgStat_Wal, stat_reset_timestamp)) != 0;
+}
+
+/*
+ * SLRU statistics counts waiting to be written to the shared activity
+ * statistics. We assume this variable inits to zeroes. Entries are
+ * one-to-one with slru_names[].
+ * Changes of SLRU counters are reported within critical sections so we use
+ * static memory in order to avoid memory allocation.
+ */
+static PgStat_SLRUStats local_SLRUStats[SLRU_NUM_ELEMENTS];
+static bool have_slrustats = false;
/* ----------
* Local data
* ----------
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
-
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
- *
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Types to define shared statistics structure.
+ *
+ * Per-object statistics are stored in a "shared stats", corresponding struct
+ * that has a header part common among all object types in DSA-allocated
+ * memory. All shared stats are pointed from a dshash via a dsa_pointer. This
+ * structure make the shared stats immovable against dshash resizing, allows a
+ * backend point to shared stats entries via a native pointer and allows
+ * locking at stats-entry level. The per-entry locking reduces lock contention
+ * compared to partition lock of dshash. A backend accumulates stats numbers in
+ * a stats entry in the local memory space then flushes the numbers to shared
+ * stats entries at basically transaction end.
+ *
+ * Each stat entry type has a fixed member PgStat_HashEntryHeader as the first
+ * element.
+ *
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStat_Stat*Entry (dsa memory block)
+ *
+ * Shared stats entries are directly pointed from pgstat_localhash hash:
+ *
+ * pgstat_localhash pgStatEntHash
+ * -> PgStatLocalHashEntry (equivalent of PgStatHashEntry)
+ * (native pointer)-> PgStat_Stat*Entry (dsa memory block)
+ *
+ * Local stats that are waiting for being flushed to share stats are stored as:
+ *
+ * pgstat_localhash pgStatLocalHash
+ * -> PgStatLocalHashEntry (local hash entry)
+ * (native pointer)-> PgStat_Stat*Entry/TableStatus (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION, /* per-function statistics */
+ PGSTAT_TYPE_REPLSLOT /* per-replication-slot statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry body size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static const size_t pgstat_sharedentsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry), /* PGSTAT_TYPE_FUNCTION */
+ sizeof(PgStat_ReplSlot) /* PGSTAT_TYPE_REPLSLOT */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static const size_t pgstat_localentsize[] =
+{
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry),/* PGSTAT_TYPE_FUNCTION */
+ sizeof(PgStat_ReplSlot) /* PGSTAT_TYPE_REPLSLOT */
+};
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * We shoud avoid overwriting header part of a shared entry. Use these macros
+ * to know what portion of the struct to be written or read. PSTAT_SHENT_BODY
+ * returns a bit smaller address than the actual address of the next member but
+ * that doesn't matter.
*/
-static HTAB *pgStatFunctions = NULL;
+#define PGSTAT_SHENT_BODY(e) (((char *)(e)) + sizeof(PgStat_StatEntryHeader))
+#define PGSTAT_SHENT_BODY_LEN(t) \
+ (pgstat_sharedentsize[t] - sizeof(PgStat_StatEntryHeader))
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object ID, either table or function. */
+} PgStatHashKey;
+
+/* struct for shared statistics hash entry */
+typedef struct PgStatHashEntry
+{
+ PgStatHashKey key; /* hash key */
+ dsa_pointer body;/* pointer to shared stats in PgStat_StatEntryHeader */
+} PgStatHashEntry;
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashKey key; /* hash key */
+ char status; /* for simplehash use */
+ PgStat_StatEntryHeader *body; /* pointer to stats body in local heap */
+ dsa_pointer dsapointer; /* dsa pointer of body */
+} PgStatLocalHashEntry;
+
+/* parameter for the shared hash */
+static const dshash_parameters dsh_params = {
+ sizeof(PgStatHashKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* define hashtable for local hashes */
+#define SH_PREFIX pgstat_localhash
+#define SH_ELEMENT_TYPE PgStatLocalHashEntry
+#define SH_KEY_TYPE PgStatHashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ hash_bytes((unsigned char *)&key, sizeof(PgStatHashKey))
+#define SH_EQUAL(tb, a, b) (memcmp(&a, &b, sizeof(PgStatHashKey)) == 0)
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * The local cache to index shared stats entries.
+ *
+ * This is a local hash to store native pointers to shared hash
+ * entries. pgStatEntHashAge is copied from StatsShmem->gc_count at creation
+ * and garbage collection.
*/
-static bool have_function_stats = false;
+static pgstat_localhash_hash *pgStatEntHash = NULL;
+static int pgStatEntHashAge = 0; /* cache age of pgStatEntHash */
+
+/* Local stats numbers are stored here. */
+static pgstat_localhash_hash *pgStatLocalHash = NULL;
+
+/* entry type for oid hash */
+typedef struct pgstat_oident
+{
+ Oid oid;
+ char status;
+} pgstat_oident;
+
+/* Define hashtable for OID hashes. */
+StaticAssertDecl(sizeof(Oid) == 4, "oid is not compatible with uint32");
+#define SH_PREFIX pgstat_oid
+#define SH_ELEMENT_TYPE pgstat_oident
+#define SH_KEY_TYPE Oid
+#define SH_KEY oid
+#define SH_HASH_KEY(tb, key) hash_bytes_uint32(key)
+#define SH_EQUAL(tb, a, b) (a == b)
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -273,11 +411,8 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -286,23 +421,9 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Make our own memory context to make it easy to track memory usage.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-static PgStat_WalStats walStats;
-static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
-static PgStat_ReplSlotStats *replSlotStats;
-static int nReplSlotStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+MemoryContext pgStatCacheContext = NULL;
/*
* Total time charged to functions so far in the current backend.
@@ -311,40 +432,57 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/* Simple caching feature for pgstatfuncs */
+static PgStatHashKey stathashkey_zero = {0};
+static PgStatHashKey cached_dbent_key = {0};
+static PgStat_StatDBEntry cached_dbent;
+static PgStatHashKey cached_tabent_key = {0};
+static PgStat_StatTabEntry cached_tabent;
+static PgStatHashKey cached_funcent_key = {0};
+static PgStat_StatFuncEntry cached_funcent;
+
+static PgStat_Archiver cached_archiverstats;
+static bool cached_archiverstats_is_valid = false;
+static PgStat_BgWriter cached_bgwriterstats;
+static bool cached_bgwriterstats_is_valid = false;
+static PgStat_CheckPointer cached_checkpointerstats;
+static bool cached_checkpointerstats_is_valid = false;
+static PgStat_Wal cached_walstats;
+static bool cached_walstats_is_valid = false;
+static PgStat_SLRUStats cached_slrustats;
+static bool cached_slrustats_is_valid = false;
+static PgStat_ReplSlot *cached_replslotstats = NULL;
+static int n_cached_replslotstats = -1;
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static void pgstat_write_statsfile(void);
+
+static void pgstat_read_statsfile(void);
static void pgstat_read_current_status(void);
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
+static PgStat_StatEntryHeader * get_stat_entry(PgStatTypes type, Oid dbid,
+ Oid objid, bool nowait,
+ bool create, bool *found);
-static int pgstat_replslot_index(const char *name, bool create_it);
-static void pgstat_reset_replslot(int i, TimestampTz ts);
+static bool flush_tabstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_funcstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_dbstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_walstat(bool nowait);
+static bool flush_slrustat(bool nowait);
+static void delete_current_stats_entry(dshash_seq_status *hstat);
+static PgStat_StatEntryHeader * get_local_stat_entry(PgStatTypes type, Oid dbid,
+ Oid objid, bool create,
+ bool *found);
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static void pgstat_send_slru(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
+static pgstat_oid_hash *collect_oids(Oid catalogid, AttrNumber anum_oid);
static void pgstat_setup_memcxt(void);
@@ -354,491 +492,645 @@ static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
-static void pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_wal(PgStat_MsgWal *msg, int len);
-static void pgstat_recv_slru(PgStat_MsgSLRU *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
+ bool found;
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(), &found);
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ ConditionVariableInit(&StatsShmem->holdoff_cv);
+ pg_atomic_init_u32(&StatsShmem->archiver_changecount, 0);
+ pg_atomic_init_u32(&StatsShmem->bgwriter_changecount, 0);
+ pg_atomic_init_u32(&StatsShmem->checkpointer_changecount, 0);
+
+ pg_atomic_init_u64(&StatsShmem->gc_count, 0);
+
+ LWLockInitialize(&StatsShmem->wal_stats_lock, LWTRANCHE_STATS);
+ }
+}
+
+/* ----------
+ * allow_next_attacher() -
+ *
+ * Let other processes to go ahead attaching the shared stats area.
+ * ----------
+ */
+static void
+allow_next_attacher(void)
+{
+ bool triggerd = false;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (StatsShmem->attach_holdoff)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
+ StatsShmem->attach_holdoff = false;
+ triggerd = true;
}
+ LWLockRelease(StatsLock);
+
+ if (triggerd)
+ ConditionVariableBroadcast(&StatsShmem->holdoff_cv);
+}
+
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read the saved statistics file if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
/*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
+ * Don't use dsm under postmaster, or when not tracking counts.
*/
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
+ pgstat_setup_memcxt();
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+ if (area)
+ return;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* stats shared memory persists for the backend lifetime */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * The first attacher backend may still reading the stats file, or the last
+ * detacher may writing it. Wait for the work to finish.
+ */
+ ConditionVariablePrepareToSleep(&StatsShmem->holdoff_cv);
+ for (;;)
+ {
+ bool hold_off;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
-
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_SHARED);
+ hold_off = StatsShmem->attach_holdoff;
+ LWLockRelease(StatsLock);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
-
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- test_byte++; /* just make sure variable is changed */
-
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /* If we get here, we have a working socket */
- break;
+ if (!hold_off)
+ break;
+
+ ConditionVariableTimedSleep(&StatsShmem->holdoff_cv, 10,
+ WAIT_EVENT_READING_STATS_FILE);
}
+ ConditionVariableCancelSleep();
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that a process going to exit can find whether it is
+ * the last one or not.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
+
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_params, 0);
+
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
+ LWLockInitialize(&StatsShmem->slru_stats.lock, LWTRANCHE_STATS);
+ pg_atomic_init_u32(&StatsShmem->slru_stats.changecount, 0);
+
+ /* Block the next attacher for a while, see the comment above. */
+ StatsShmem->attach_holdoff = true;
+
+ StatsShmem->refcount = 1;
}
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
+ LWLockRelease(StatsLock);
+
+ if (area)
{
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- ereport(LOG,
- (errmsg("getsockopt(%s) failed: %m", "SO_RCVBUF")));
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- ereport(LOG,
- (errmsg("setsockopt(%s) failed: %m", "SO_RCVBUF")));
- }
+ /*
+ * We're the first attacher process, read stats file while blocking
+ * successors.
+ */
+ Assert(StatsShmem->attach_holdoff);
+ pgstat_read_statsfile();
+ allow_next_attacher();
+ }
+ else
+ {
+ /* We're not the first one, attach existing shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_params,
+ StatsShmem->hash_handle, 0);
}
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+}
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
+/* ----------
+ * cleanup_dropped_stats_entries() -
+ * Clean up shared stats entries no longer used.
+ *
+ * Shared stats entries for dropped objects may be left referenced. Clean up
+ * our reference and drop the shared entry if needed.
+ * ----------
+ */
+static void
+cleanup_dropped_stats_entries(void)
+{
+ pgstat_localhash_iterator i;
+ PgStatLocalHashEntry *ent;
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+ if (pgStatEntHash == NULL)
+ return;
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
+ pgstat_localhash_start_iterate(pgStatEntHash, &i);
+ while ((ent = pgstat_localhash_iterate(pgStatEntHash, &i))
+ != NULL)
+ {
+ /*
+ * Free the shared memory chunk for the entry if we were the last
+ * referrer to a dropped entry.
+ */
+ if (pg_atomic_sub_fetch_u32(&ent->body->refcount, 1) < 1 &&
+ ent->body->dropped)
+ dsa_free(area, ent->dsapointer);
+ }
/*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
+ * This function is expected to be called during backend exit. So we don't
+ * bother destroying pgStatEntHash.
*/
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ pgStatEntHash = NULL;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_file)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ bool is_last_detacher = 0;
+
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
+
+ /* We shouldn't leave a reference to shared stats. */
+ cleanup_dropped_stats_entries();
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
+ /*
+ * If we are the last detacher, hold off the next attacher (if possible)
+ * until we finish writing stats file.
+ */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount == 0)
{
- int nchars;
- Oid tmp_oid;
+ StatsShmem->attach_holdoff = true;
+ is_last_detacher = true;
+ }
+ LWLockRelease(StatsLock);
- /*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
- */
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
-
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ if (is_last_detacher)
+ {
+ if (write_file)
+ pgstat_write_statsfile();
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ /* allow the next attacher, if any */
+ allow_next_attacher();
}
- FreeDir(dir);
+
+ /*
+ * Detach the area. It is automatically destroyed when the last process
+ * detached it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+
+ /* We are going to exit. Don't bother destroying local hashes. */
+ pgStatLocalHash = NULL;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
+
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
+
+ /*
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
+ */
+ detach_shared_stats(false);
+ attach_shared_stats();
}
-#ifdef EXEC_BACKEND
/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
+ * fetch_lock_statentry - common helper function to fetch and lock a stats
+ * entry for flush_tabstat, flush_funcstat and flush_dbstat.
*/
-static pid_t
-pgstat_forkexec(void)
+static PgStat_StatEntryHeader *
+fetch_lock_statentry(PgStatTypes type, Oid dboid, Oid objid, bool nowait)
{
- char *av[10];
- int ac = 0;
+ PgStat_StatEntryHeader *header;
+
+ /* find shared table stats entry corresponding to the local entry */
+ header = (PgStat_StatEntryHeader *)
+ get_stat_entry(type, dboid, objid, nowait, true, NULL);
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ /* skip if dshash failed to acquire lock */
+ if (header == NULL)
+ return false;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ return false;
- return postmaster_forkexec(ac, av);
+ return header;
}
-#endif /* EXEC_BACKEND */
-/*
- * pgstat_start() -
+/* ----------
+ * get_stat_entry() -
*
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new base entry. If found is not NULL, it is set to true
+ * if existing entry is found or false if not.
+ * ----------
+ */
+static PgStat_StatEntryHeader *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait, bool create,
+ bool *found)
+{
+ PgStatHashEntry *shhashent;
+ PgStatLocalHashEntry *lohashent;
+ PgStat_StatEntryHeader *shheader = NULL;
+ PgStatHashKey key;
+ bool shfound;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ if (pgStatEntHash)
+ {
+ uint64 currage;
+
+ /*
+ * pgStatEntHashAge increments quite slowly than the time the following
+ * loop takes so this is expected to iterate no more than twice.
+ */
+ while (unlikely
+ (pgStatEntHashAge !=
+ (currage = pg_atomic_read_u64(&StatsShmem->gc_count))))
+ {
+ pgstat_localhash_iterator i;
+
+ /*
+ * Some entries have been dropped. Invalidate cache pointer to
+ * them.
+ */
+ pgstat_localhash_start_iterate(pgStatEntHash, &i);
+ while ((lohashent = pgstat_localhash_iterate(pgStatEntHash, &i))
+ != NULL)
+ {
+ PgStat_StatEntryHeader *header = lohashent->body;
+ if (header->dropped)
+ {
+ pgstat_localhash_delete(pgStatEntHash, key);
+
+ if (pg_atomic_sub_fetch_u32(&header->refcount, 1) < 1)
+ {
+ /*
+ * We're the last referrer to this entry, drop the
+ * shared entry.
+ */
+ dsa_free(area, lohashent->dsapointer);
+ }
+ }
+ }
+
+ pgStatEntHashAge = currage;
+ }
+
+ lohashent = pgstat_localhash_lookup(pgStatEntHash, key);
+
+ if (lohashent)
+ {
+ if (found)
+ *found = true;
+ return lohashent->body;
+ }
+ }
+
+ shhashent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &shfound);
+ if (shhashent)
+ {
+ if (create && !shfound)
+ {
+ /* Create new stats entry. */
+ dsa_pointer chunk = dsa_allocate0(area,
+ pgstat_sharedentsize[type]);
+
+ shheader = dsa_get_address(area, chunk);
+ LWLockInitialize(&shheader->lock, LWTRANCHE_STATS);
+ pg_atomic_init_u32(&shheader->refcount, 0);
+
+ /* Link the new entry from the hash entry. */
+ shhashent->body = chunk;
+ }
+ else
+ shheader = dsa_get_address(area, shhashent->body);
+
+ /*
+ * We expose this shared entry now. You might think that the entry can
+ * be removed by a concurrent backend, but since we are creating an
+ * stats entry, the object actually exists and used in the upper
+ * layer. Such an object cannot be dropped until the first vacuum after
+ * the current transaction ends.
+ */
+ dshash_release_lock(pgStatSharedHash, shhashent);
+
+ /* register to local hash if possible */
+ if (pgStatEntHash || pgStatCacheContext)
+ {
+ bool lofound;
+
+ if (pgStatEntHash == NULL)
+ {
+ pgStatEntHash =
+ pgstat_localhash_create(pgStatCacheContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
+ pgStatEntHashAge =
+ pg_atomic_read_u64(&StatsShmem->gc_count);
+ }
+
+ lohashent =
+ pgstat_localhash_insert(pgStatEntHash, key, &lofound);
+
+ Assert(!lofound);
+ lohashent->body = shheader;
+ lohashent->dsapointer = shhashent->body;
+
+ pg_atomic_add_fetch_u32(&shheader->refcount, 1);
+ }
+ }
+
+ if (found)
+ *found = shfound;
+
+ return shheader;
+}
+
+/*
+ * flush_walstat - flush out a local WAL stats entries
*
- * Returns PID of child process, or 0 if fail.
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
*
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Returns true if all local WAL stats are successfully flushed out.
*/
-int
-pgstat_start(void)
+static bool
+flush_walstat(bool nowait)
{
- time_t curtime;
- pid_t pgStatPid;
+ PgStat_Wal *s = &StatsShmem->wal_stats;
+ PgStat_Wal *l = &WalStats;
+ WalUsage all_zeroes = {0} PG_USED_FOR_ASSERTS_ONLY;
+
+ /*
+ * We don't update the WAL usage portion of the local WalStats
+ * elsewhere. Instead, fill in that portion with the difference of
+ * pgWalUsage since the previous call.
+ */
+ Assert(memcmp(&l->wal_usage, &all_zeroes, sizeof(WalUsage)) == 0);
+ WalUsageAccumDiff(&l->wal_usage, &pgWalUsage, &prevWalUsage);
/*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
+ * This function can be called even if nothing at all has happened. Avoid
+ * taking lock for nothing in that case.
*/
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
+ if (!walstats_pending())
+ return true;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&StatsShmem->wal_stats_lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&StatsShmem->wal_stats_lock,
+ LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ s->wal_usage.wal_records += l->wal_usage.wal_records;
+ s->wal_usage.wal_fpi += l->wal_usage.wal_fpi;
+ s->wal_usage.wal_bytes += l->wal_usage.wal_bytes;
+ s->wal_buffers_full += l->wal_buffers_full;
+ LWLockRelease(&StatsShmem->wal_stats_lock);
/*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
+ * Save the current counters for the subsequent calculation of WAL usage.
*/
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ prevWalUsage = pgWalUsage;
/*
- * Okay, fork off the collector.
+ * Clear out the statistics buffer, so it can be re-used.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
+ MemSet(&WalStats, 0, sizeof(WalStats));
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
+ return true;
+}
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+/*
+ * flush_slrustat - flush out a local SLRU stats entries
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true. Writer processes are mutually excluded
+ * using LWLock, but readers are expected to use change-count protocol to avoid
+ * interference with writers.
+ *
+ * Returns true if all local SLRU stats are successfully flushed out.
+ */
+static bool
+flush_slrustat(bool nowait)
+{
+ uint32 assert_changecount PG_USED_FOR_ASSERTS_ONLY;
+ int i;
+
+ if (!have_slrustats)
+ return true;
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&StatsShmem->slru_stats.lock,
+ LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
- PgstatCollectorMain(0, NULL);
- break;
-#endif
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_stats.changecount, 1);
+ Assert((assert_changecount & 1) == 0);
- default:
- return (int) pgStatPid;
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS ; i++)
+ {
+ PgStat_SLRUStats *sharedent = &StatsShmem->slru_stats.entry[i];
+ PgStat_SLRUStats *localent = &local_SLRUStats[i];
+
+ sharedent->blocks_zeroed += localent->blocks_zeroed;
+ sharedent->blocks_hit += localent->blocks_hit;
+ sharedent->blocks_read += localent->blocks_read;
+ sharedent->blocks_written += localent->blocks_written;
+ sharedent->blocks_exists += localent->blocks_exists;
+ sharedent->flush += localent->flush;
+ sharedent->truncate += localent->truncate;
}
- /* shouldn't get here */
- return 0;
+ /* done, clear the local entry */
+ MemSet(local_SLRUStats, 0,
+ sizeof(PgStat_SLRUStats) * SLRU_NUM_ELEMENTS);
+
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_stats.changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+
+ have_slrustats = false;
+
+ return true;
+}
+
+/* ----------
+ * delete_current_stats_entry()
+ *
+ * Deletes the given shared entry from shared stats hash. The entry must be
+ * exclusively locked.
+ * ----------
+ */
+static void
+delete_current_stats_entry(dshash_seq_status *hstat)
+{
+ dsa_pointer pdsa;
+ PgStat_StatEntryHeader *header;
+ PgStatHashEntry *ent;
+
+ ent = dshash_get_current(hstat);
+ pdsa = ent->body;
+ header = dsa_get_address(area, pdsa);
+
+ /* No one find this entry ever after. */
+ dshash_delete_current(hstat);
+
+ /*
+ * Let the referrers drop the entry if any. Refcount won't be decremented
+ * until "dropped" is set true and StatsShmem->gc_count is incremented
+ * later. So we can check refcount to set dropped without holding a
+ * lock. If no one is referring this entry, free it immediately.
+ */
+
+ if (pg_atomic_read_u32(&header->refcount) > 0)
+ header->dropped = true;
+ else
+ dsa_free(area, pdsa);
+
+ return;
}
-void
-allow_immediate_pgstat_restart(void)
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize the statsbody part of the returned
+ * memory.
+ * ----------
+ */
+static PgStat_StatEntryHeader *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- last_pgstat_start_time = 0;
+ PgStatHashKey key;
+ PgStatLocalHashEntry *entry;
+
+ if (pgStatLocalHash == NULL)
+ pgStatLocalHash = pgstat_localhash_create(pgStatCacheContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ if (create)
+ entry = pgstat_localhash_insert(pgStatLocalHash, key, found);
+ else
+ entry = pgstat_localhash_lookup(pgStatLocalHash, key);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ entry->body = MemoryContextAllocZero(TopMemoryContext,
+ pgstat_localentsize[type]);
+
+ return entry->body;
}
/* ------------------------------------------------------------
@@ -846,150 +1138,399 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait;
int i;
+ uint64 oldval;
+
+ /* Return if not active */
+ if (area == NULL)
+ return 0;
+
+ /*
+ * We need a database entry if the following stats exists.
+ */
+ if (pgStatXactCommit > 0 || pgStatXactRollback > 0 ||
+ pgStatBlockReadTime > 0 || pgStatBlockWriteTime > 0)
+ get_local_dbstat_entry(MyDatabaseId);
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (pgStatLocalHash == NULL && have_slrustats && !walstats_pending())
+ return 0;
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
-
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
-
- /*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
- */
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+
+ if (!force)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
-
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
-
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ force = true;
+ }
+
+ /* don't wait for lock acquisition when !force */
+ nowait = !force;
+
+ if (pgStatLocalHash)
+ {
+ int remains = 0;
+ pgstat_localhash_iterator i;
+ List *dbentlist = NIL;
+ ListCell *lc;
+ PgStatLocalHashEntry *lent;
+
+ /* Step 1: flush out other than database stats */
+ pgstat_localhash_start_iterate(pgStatLocalHash, &i);
+ while ((lent = pgstat_localhash_iterate(pgStatLocalHash, &i)) != NULL)
+ {
+ bool remove = false;
+
+ switch (lent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /*
+ * flush_tabstat applies some of stats numbers of flushed
+ * entries into local database stats. Just remember the
+ * database entries for now then flush-out them later.
+ */
+ dbentlist = lappend(dbentlist, lent);
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_REPLSLOT:
+ /* We don't have that kind of local entry */
+ Assert(false);
+ }
+
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->body);
+ lent->body = NULL;
+ pgstat_localhash_delete(pgStatLocalHash, lent->key);
+ }
+
+ /* Step 2: flush out database stats */
+ foreach(lc, dbentlist)
+ {
+ PgStatLocalHashEntry *lent = (PgStatLocalHashEntry *) lfirst(lc);
+
+ if (flush_dbstat(lent, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->body);
+ lent->body = NULL;
+ pgstat_localhash_delete(pgStatLocalHash, lent->key);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ list_free(dbentlist);
+ dbentlist = NULL;
+
+ if (remains <= 0)
+ {
+ pgstat_localhash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
+ }
+
+ /* flush wal stats */
+ flush_walstat(nowait);
+
+ /* flush SLRU stats */
+ flush_slrustat(nowait);
+
+ /*
+ * Publish the time of the last flush, but we don't notify the change of
+ * the timestamp itself. Readers will get sufficiently recent timestamp.
+ * If we failed to update the value, concurrent processes should have
+ * updated it to sufficiently recent time.
+ *
+ * XXX: The loop might be unnecessary for the reason above.
+ */
+ oldval = pg_atomic_read_u64(&StatsShmem->stats_timestamp);
+
+ for (i = 0 ; i < 10 ; i++)
+ {
+ if (oldval >= now ||
+ pg_atomic_compare_exchange_u64(&StatsShmem->stats_timestamp,
+ &oldval, (uint64) now))
+ break;
}
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * Some of the local stats may have not been flushed due to lock
+ * contention. If we have such pending local stats here, let the caller
+ * know the retry interval.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
+ if (pgStatLocalHash != NULL || have_slrustats || walstats_pending())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
- /* Send WAL statistics */
- pgstat_send_wal();
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
- /* Finally send SLRU statistics */
- pgstat_send_slru();
+ return retry_interval / 1000;
+ }
+
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
}
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
*/
-static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+static bool
+flush_tabstat(PgStatLocalHashEntry *ent, bool nowait)
{
- int n;
- int len;
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+
+ Assert(ent->key.type == PGSTAT_TYPE_TABLE);
+ lstats = (PgStat_TableStatus *) ent->body;
+ dboid = ent->key.databaseid;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ {
+ /* This local entry is going to be dropped, delink from relcache. */
+ pgstat_delinkstats(lstats->relation);
+ return true;
+ }
+
+ /* find shared table stats entry corresponding to the local entry */
+ shtabstats = (PgStat_StatTabEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_TABLE, dboid, ent->key.objectid,
+ nowait);
+
+ if (shtabstats == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->inserts_since_vacuum += lstats->t_counts.t_tuples_inserted;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shtabstats->header.lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /* This local entry is going to be dropped, delink from relcache. */
+ pgstat_delinkstats(lstats->relation);
+
+ return true;
+}
+
+/*
+ * flush_funcstat - flush out a local function stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatLocalHashEntry *ent, bool nowait)
+{
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStat_StatFuncEntry *shfuncent = NULL; /* shared stats entry */
+
+ Assert(ent->key.type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) ent->body;
+
+ /* localent always has non-zero content */
+
+ /* find shared table stats entry corresponding to the local entry */
+ shfuncent = (PgStat_StatFuncEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ ent->key.objectid, nowait);
+ if (shfuncent == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ shfuncent->f_numcalls += localent->f_counts.f_numcalls;
+ shfuncent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ shfuncent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shfuncent->header.lock);
+
+ return true;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatLocalHashEntry *ent, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(ent->key.type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &ent->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ sharedent = (PgStat_StatDBEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_DB, ent->key.databaseid, InvalidOid,
+ nowait);
+
+ if (!sharedent)
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(ent->key.databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -997,281 +1538,138 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&sharedent->header.lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
-/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
- */
-static void
-pgstat_send_funcstats(void)
-{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
- {
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
-
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
-}
-
-
/* ----------
* pgstat_vacuum_stat() -
*
- * Will tell the collector about objects he can get rid of.
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
* ----------
*/
void
pgstat_vacuum_stat(void)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
+ pgstat_oid_hash *dbids; /* database ids */
+ pgstat_oid_hash *relids; /* relation ids in the current database */
+ pgstat_oid_hash *funcids;/* function ids in the current database */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
return;
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
+ nvictims = 0;
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ /* some of the dshash entries are to be removed, take exclusive lock. */
+ dshash_seq_init(&dshstat, pgStatSharedHash, true);
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
+ pgstat_oid_hash *oidhash;
+ Oid key;
CHECK_FOR_INTERRUPTS();
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
+ switch (ent->key.type)
{
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidhash = dbids;
+ key = ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidhash = relids;
+ key = ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidhash = funcids;
+ key = ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_REPLSLOT:
+ /*
+ * We don't bother vacuumming this kind of entries because the
+ * number of entries is quite small and entries are likely to
+ * be reused soon.
+ */
continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
}
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
+ /* Skip existent objects. */
+ if (pgstat_oid_lookup(oidhash, key) != NULL)
+ continue;
- hash_destroy(htab);
+ /* drop this etnry */
+ delete_current_stats_entry(&dshstat);
+ nvictims++;
}
+ dshash_seq_term(&dshstat);
+ pgstat_oid_destroy(dbids);
+ pgstat_oid_destroy(relids);
+ pgstat_oid_destroy(funcids);
+
+ if (nvictims > 0)
+ pg_atomic_add_fetch_u64(&StatsShmem->gc_count, 1);
}
-
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
- * into a temporary hash table. Caller should hash_destroy the result
+ * into a temporary hash table. Caller should pgsstat_oid_destroy the result
* when done with it. (However, we make the table in CurrentMemoryContext
* so that it will be freed properly in event of an error.)
* ----------
*/
-static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+static pgstat_oid_hash *
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
- HTAB *htab;
- HASHCTL hash_ctl;
+ pgstat_oid_hash *rethash;
Relation rel;
TableScanDesc scan;
HeapTuple tup;
Snapshot snapshot;
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(Oid);
- hash_ctl.hcxt = CurrentMemoryContext;
- htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ rethash = pgstat_oid_create(CurrentMemoryContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
rel = table_open(catalogid, AccessShareLock);
snapshot = RegisterSnapshot(GetLatestSnapshot());
@@ -1280,81 +1678,61 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
{
Oid thisoid;
bool isnull;
+ bool found;
thisoid = heap_getattr(tup, anum_oid, RelationGetDescr(rel), &isnull);
Assert(!isnull);
CHECK_FOR_INTERRUPTS();
- (void) hash_search(htab, (void *) &thisoid, HASH_ENTER, NULL);
+ pgstat_oid_insert(rethash, thisoid, &found);
}
table_endscan(scan);
UnregisterSnapshot(snapshot);
table_close(rel, AccessShareLock);
- return htab;
+ return rethash;
}
-
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ Assert(OidIsValid(databaseid));
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
-
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ /* some of the dshash entries are to be removed, take exclusive lock. */
+ dshash_seq_init(&hstat, pgStatSharedHash, true);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.databaseid == MyDatabaseId)
+ delete_current_stats_entry(&hstat);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Let readers run a garbage collection of local hashes */
+ pg_atomic_add_fetch_u64(&StatsShmem->gc_count, 1);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1363,53 +1741,146 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatEntryHeader *header;
+
+ if (p->key.databaseid != MyDatabaseId)
+ continue;
+
+ header = dsa_get_address(area, p->body);
+
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ memset(PGSTAT_SHENT_BODY(header), 0,
+ PGSTAT_SHENT_BODY_LEN(p->key.type));
+
+ if (p->key.type == PGSTAT_TYPE_DB)
+ {
+ PgStat_StatDBEntry *dbstat = (PgStat_StatDBEntry *) header;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ }
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Invalidate the simple cache keys */
+ cached_dbent_key = stathashkey_zero;
+ cached_tabent_key = stathashkey_zero;
+ cached_funcent_key = stathashkey_zero;
+}
+
+/*
+ * pgstat_copy_global_stats - helper function for functions
+ * pgstat_fetch_stat_*() and pgstat_reset_shared_counters().
+ *
+ * Copies out the specified memory area following change-count protocol.
+ */
+static inline void
+pgstat_copy_global_stats(void *dst, void*src, size_t len,
+ pg_atomic_uint32 *count)
+{
+ int before_changecount;
+ int after_changecount;
+
+ after_changecount = pg_atomic_read_u32(count);
+
+ do
+ {
+ before_changecount = after_changecount;
+ memcpy(dst, src, len);
+ after_changecount = pg_atomic_read_u32(count);
+ } while ((before_changecount & 1) == 1 ||
+ after_changecount != before_changecount);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
+ *
+ * We don't scribble on shared stats while resetting to avoid locking on
+ * shared stats struct. Instead, just record the current counters in another
+ * shared struct, which is protected by StatsLock. See
+ * pgstat_fetch_stat_(archiver|bgwriter|checkpointer) for the reader side.
* ----------
*/
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ TimestampTz now = GetCurrentTimestamp();
+ PgStat_Shared_Reset_Target t;
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ t = RESET_ARCHIVER;
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ t = RESET_BGWRITER;
else if (strcmp(target, "wal") == 0)
- msg.m_resettarget = RESET_WAL;
+ t = RESET_WAL;
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\", \"bgwriter\" or \"wal\".")));
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
+ /* Reset statistics for the cluster. */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ switch (t)
+ {
+ case RESET_ARCHIVER:
+ pgstat_copy_global_stats(&StatsShmem->archiver_reset_offset,
+ &StatsShmem->archiver_stats,
+ sizeof(PgStat_Archiver),
+ &StatsShmem->archiver_changecount);
+ StatsShmem->archiver_reset_offset.stat_reset_timestamp = now;
+ cached_archiverstats_is_valid = false;
+ break;
+
+ case RESET_BGWRITER:
+ pgstat_copy_global_stats(&StatsShmem->bgwriter_reset_offset,
+ &StatsShmem->bgwriter_stats,
+ sizeof(PgStat_BgWriter),
+ &StatsShmem->bgwriter_changecount);
+ pgstat_copy_global_stats(&StatsShmem->checkpointer_reset_offset,
+ &StatsShmem->checkpointer_stats,
+ sizeof(PgStat_CheckPointer),
+ &StatsShmem->checkpointer_changecount);
+ StatsShmem->bgwriter_reset_offset.stat_reset_timestamp = now;
+ cached_bgwriterstats_is_valid = false;
+ cached_checkpointerstats_is_valid = false;
+ break;
+
+ case RESET_WAL:
+ /*
+ * Differntly from the two above, WAL statistics has many writer
+ * processes and protected by wal_stats_lock.
+ */
+ LWLockAcquire(&StatsShmem->wal_stats_lock, LW_EXCLUSIVE);
+ MemSet(&StatsShmem->wal_stats, 0, sizeof(PgStat_Wal));
+ StatsShmem->wal_stats.stat_reset_timestamp = now;
+ LWLockRelease(&StatsShmem->wal_stats_lock);
+ cached_walstats_is_valid = false;
+ break;
+ }
+
+ LWLockRelease(StatsLock);
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1418,17 +1889,37 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStat_StatEntryHeader *header;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ dbentry = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid, false, false,
+ NULL);
+ Assert(dbentry);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&dbentry->header.lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&dbentry->header.lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ header = get_stat_entry(stattype, MyDatabaseId, objoid, false, false, NULL);
+
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ memset(PGSTAT_SHENT_BODY(header), 0, PGSTAT_SHENT_BODY_LEN(stattype));
+ LWLockRelease(&header->lock);
}
/* ----------
@@ -1444,15 +1935,40 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_reset_slru_counter(const char *name)
{
- PgStat_MsgResetslrucounter msg;
+ int i;
+ TimestampTz ts = GetCurrentTimestamp();
+ uint32 assert_changecount;PG_USED_FOR_ASSERTS_ONLY;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ if (name)
+ {
+ i = pgstat_slru_index(name);
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_changecount, 1);
+ Assert((assert_changecount & 1) == 0);
+ MemSet(&StatsShmem->slru_stats.entry[i], 0,
+ sizeof(PgStat_SLRUStats));
+ StatsShmem->slru_stats.entry[i].stat_reset_timestamp = ts;
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+ }
+ else
+ {
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_changecount, 1);
+ Assert((assert_changecount & 1) == 0);
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS; i++)
+ {
+ MemSet(&StatsShmem->slru_stats.entry[i], 0,
+ sizeof(PgStat_SLRUStats));
+ StatsShmem->slru_stats.entry[i].stat_reset_timestamp = ts;
+ }
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSLRUCOUNTER);
- msg.m_index = (name) ? pgstat_slru_index(name) : -1;
-
- pgstat_send(&msg, sizeof(msg));
+ cached_slrustats_is_valid = false;
}
/* ----------
@@ -1468,20 +1984,19 @@ pgstat_reset_slru_counter(const char *name)
void
pgstat_reset_replslot_counter(const char *name)
{
- PgStat_MsgResetreplslotcounter msg;
+ int startidx;
+ int endidx;
+ int i;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
if (name)
{
ReplicationSlot *slot;
-
- /*
- * Check if the slot exits with the given name. It is possible that by
- * the time this message is executed the slot is dropped but at least
- * this check will ensure that the given name is for a valid slot.
- */
+
+ /* Check if the slot exits with the given name. */
LWLockAcquire(ReplicationSlotControlLock, LW_SHARED);
slot = SearchNamedReplicationSlot(name);
LWLockRelease(ReplicationSlotControlLock);
@@ -1499,15 +2014,36 @@ pgstat_reset_replslot_counter(const char *name)
if (SlotIsPhysical(slot))
return;
- strlcpy(msg.m_slotname, name, NAMEDATALEN);
- msg.clearall = false;
+ /* reset this one entry */
+ startidx = endidx = slot - ReplicationSlotCtl->replication_slots;
}
else
- msg.clearall = true;
+ {
+ /* reset all existent entries */
+ startidx = 0;
+ endidx = max_replication_slots - 1;
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETREPLSLOTCOUNTER);
+ ts = GetCurrentTimestamp();
+ for (i = startidx ; i <= endidx ; i++)
+ {
+ PgStat_ReplSlot *shent;
- pgstat_send(&msg, sizeof(msg));
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, false, NULL);
+
+ /* Skip non-existent entries */
+ if (!shent)
+ continue;
+
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ memset(&shent->spill_txns, 0,
+ offsetof(PgStat_ReplSlot, stat_reset_timestamp) -
+ offsetof(PgStat_ReplSlot, spill_txns));
+ shent->stat_reset_timestamp = ts;
+ LWLockRelease(&shent->header.lock);
+ }
}
/* ----------
@@ -1521,48 +2057,93 @@ pgstat_reset_replslot_counter(const char *name)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ /*
+ * End-of-vacuum is reported instantly. Report the start the same way for
+ * consistency. Vacuum doesn't run frequently and is a long-lasting
+ * operation so it doesn't matter if we get blocked here a little.
+ */
+ dbentry = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, dboid, InvalidOid, false, true, NULL);
- pgstat_send(&msg, sizeof(msg));
+ ts = GetCurrentTimestamp();;
+ LWLockAcquire(&dbentry->header.lock, LW_EXCLUSIVE);
+ dbentry->last_autovac_time = ts;
+ LWLockRelease(&dbentry->header.lock);
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+
+ /*
+ * Differently from ordinary operations, maintenance commands take longer
+ * time and getting blocked at the end of work doesn't matter. Furthermore,
+ * this can prevent the stats updates made by the transactions that ends
+ * after this vacuum from being canceled by a delayed vacuum report.
+ * Update shared stats entry directly for the above reasons.
+ */
+ tabentry = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, tableoid, false, true, NULL);
+
+ LWLockAcquire(&tabentry->header.lock, LW_EXCLUSIVE);
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * It is quite possible that a non-aggressive VACUUM ended up skipping
+ * various pages, however, we'll zero the insert counter here regardless.
+ * It's currently used only to track when we need to perform an "insert"
+ * autovacuum, which are mainly intended to freeze newly inserted tuples.
+ * Zeroing this may just mean we'll not try to vacuum the table again
+ * until enough tuples have been inserted to trigger another insert
+ * autovacuum. An anti-wraparound autovacuum will catch any persistent
+ * stragglers.
+ */
+ tabentry->inserts_since_vacuum = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->vacuum_count++;
+ }
+
+ LWLockRelease(&tabentry->header.lock);
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1573,9 +2154,11 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId);
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1583,10 +2166,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1604,137 +2187,223 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Differently from ordinary operations, maintenance commands take longer
+ * time and getting blocked at the end of work doesn't matter. Furthermore,
+ * this can prevent the stats updates made by the transactions that ends
+ * after this analyze from being canceled by a delayed analyze report.
+ * Update shared stats entry directly for the above reasons.
+ */
+ tabentry = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, RelationGetRelid(rel),
+ false, true, NULL);
+
+ LWLockAcquire(&tabentry->header.lock, LW_EXCLUSIVE);
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+ LWLockRelease(&tabentry->header.lock);
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
+
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
-
-
/* --------
- * pgstat_report_checksum_failures_in_db() -
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Tell the collector about one or more checksum failures.
+ * Reports about one or more checksum failures.
* --------
*/
void
pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgChecksumFailure msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
+ dbentry = get_local_dbstat_entry(dboid);
- pgstat_send(&msg, sizeof(msg));
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
/* ----------
* pgstat_report_replslot() -
*
- * Tell the collector about replication slot statistics.
+ * Report replication slot activity.
* ----------
*/
void
-pgstat_report_replslot(const char *slotname, int spilltxns, int spillcount,
- int spillbytes, int streamtxns, int streamcount, int streambytes)
+pgstat_report_replslot(const char *slotname,
+ int spilltxns, int spillcount, int spillbytes,
+ int streamtxns, int streamcount, int streambytes)
{
- PgStat_MsgReplSlot msg;
+ PgStat_ReplSlot *shent;
+ int i;
+ bool found;
+
+ if (!area)
+ return;
+
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ if (strcmp(NameStr(ReplicationSlotCtl->replication_slots[i].data.name),
+ slotname) == 0)
+ break;
+
+ }
/*
- * Prepare and send the message
+ * the slot should have been removed. just ignore it. We create the entry
+ * for the slot with this name next time.
*/
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_REPLSLOT);
- strlcpy(msg.m_slotname, slotname, NAMEDATALEN);
- msg.m_drop = false;
- msg.m_spill_txns = spilltxns;
- msg.m_spill_count = spillcount;
- msg.m_spill_bytes = spillbytes;
- msg.m_stream_txns = streamtxns;
- msg.m_stream_count = streamcount;
- msg.m_stream_bytes = streambytes;
- pgstat_send(&msg, sizeof(PgStat_MsgReplSlot));
+ if (i == max_replication_slots)
+ return;
+
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, true, &found);
+
+ /* Clear the counters and reset dropped when we reuse it */
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ if (shent->header.dropped || !found)
+ {
+ memset(&shent->spill_txns, 0,
+ sizeof(PgStat_ReplSlot) - offsetof(PgStat_ReplSlot, spill_txns));
+ strlcpy(shent->slotname, slotname, NAMEDATALEN);
+ shent->header.dropped = false;
+ }
+
+ shent->spill_txns += spilltxns;
+ shent->spill_count += spillcount;
+ shent->spill_bytes += spillbytes;
+ shent->stream_txns += streamtxns;
+ shent->stream_count += streamcount;
+ shent->stream_bytes += streambytes;
+ LWLockRelease(&shent->header.lock);
}
/* ----------
@@ -1746,55 +2415,44 @@ pgstat_report_replslot(const char *slotname, int spilltxns, int spillcount,
void
pgstat_report_replslot_drop(const char *slotname)
{
- PgStat_MsgReplSlot msg;
+ int i;
+ PgStat_ReplSlot *shent;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_REPLSLOT);
- strlcpy(msg.m_slotname, slotname, NAMEDATALEN);
- msg.m_drop = true;
- pgstat_send(&msg, sizeof(PgStat_MsgReplSlot));
-}
+ Assert(area);
+ if (!area)
+ return;
-/* ----------
- * pgstat_ping() -
- *
- * Send some junk data to the collector to increase traffic.
- * ----------
- */
-void
-pgstat_ping(void)
-{
- PgStat_MsgDummy msg;
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ if (strcmp(NameStr(ReplicationSlotCtl->replication_slots[i].data.name),
+ slotname) == 0)
+ break;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ /* XXX: maybe the slot has been removed. just ignore it. */
+ if (i == max_replication_slots)
+ return;
+
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, false, NULL);
+
+ if (shent && !shent->header.dropped)
+ {
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ shent->header.dropped = true;
+ LWLockRelease(&shent->header.lock);
+ }
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
@@ -1809,24 +2467,9 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
- if (!found)
- MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent = (PgStat_BackendFunctionEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
fcu->fs = &htabent->f_counts;
@@ -1840,31 +2483,37 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
- return NULL;
+ PgStat_BackendFunctionEntry *ent;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ ent = (PgStat_BackendFunctionEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+
+ return ent;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1905,9 +2554,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1919,8 +2565,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1936,7 +2581,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1947,120 +2593,60 @@ pgstat_initstats(Relation rel)
* If we already set up this relation in the current transaction, nothing
* to do.
*/
- if (rel->pgstat_info != NULL &&
- rel->pgstat_info->t_id == rel_id)
+ if (rel->pgstat_info != NULL)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ /* mark this relation as the owner */
+
+ /* don't allow link a stats to multiple relcache entries */
+ Assert (rel->pgstat_info->relation == NULL);
+ rel->pgstat_info->relation = rel;
}
/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+ * pgstat_delinkstats() -
+ *
+ * Break the mutual link between a relcache entry and a local stats entry.
+ * This must be called always when one end of the link is removed.
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+void
+pgstat_delinkstats(Relation rel)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
-
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ /* remove the link to stats info if any */
+ if (rel && rel->pgstat_info)
{
- HASHCTL ctl;
-
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ /* ilnk sanity check */
+ Assert (rel->pgstat_info->relation == rel);
+ rel->pgstat_info->relation = NULL;
+ rel->pgstat_info = NULL;
}
-
- /*
- * Find an entry or create a new one.
- */
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
- if (!found)
- {
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
- }
-
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
-
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
-
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
-
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
-
- return entry;
}
/*
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel_id in the current
+ * database. If not found, try finding from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry found, return NULL, don't create a new one
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStat_TableStatus *ent;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ ent = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!ent)
+ ent = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return ent;
}
/*
@@ -2475,8 +3061,6 @@ AtPrepare_PgStat(void)
record.inserted_pre_trunc = trans->inserted_pre_trunc;
record.updated_pre_trunc = trans->updated_pre_trunc;
record.deleted_pre_trunc = trans->deleted_pre_trunc;
- record.t_id = tabstat->t_id;
- record.t_shared = tabstat->t_shared;
record.t_truncated = trans->truncated;
RegisterTwoPhaseRecord(TWOPHASE_RM_PGSTAT_ID, 0,
@@ -2491,8 +3075,8 @@ AtPrepare_PgStat(void)
*
* All we need do here is unlink the transaction stats state from the
* nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
+ * reported to the activity stats facility immediately, while the effects on
+ * live and dead tuple counts are preserved in the 2PC state file.
*
* Note: AtEOXact_PgStat is not called during PREPARE.
*/
@@ -2537,7 +3121,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2573,7 +3157,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2593,85 +3177,138 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends in a palloc'ed memory.
+ *
+ * The returned entry is stored in static memory so the content is valid until
+ * the next call of the same function for the different database.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ PgStat_StatDBEntry *shent;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for InvalidOid */
+ Assert(dbid != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_dbent_key.databaseid == dbid)
+ return &cached_dbent;
+
+ shent = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid, true, false, NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_dbent, shent, sizeof(PgStat_StatDBEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_dbent_key.databaseid = dbid;
+
+ return &cached_dbent;
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_extended(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_extended(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_extended() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is stored
+ * in static memory so the content is valid until the next call of the same
+ * function for the different table.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_extended(bool shared, Oid reloid)
+{
+ PgStat_StatTabEntry *shent;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for the InvalidOid */
+ Assert(reloid != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_tabent_key.databaseid == dboid &&
+ cached_tabent_key.objectid == reloid)
+ return &cached_tabent;
+
+ shent = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, reloid, true, false, NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_tabent, shent, sizeof(PgStat_StatTabEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_tabent_key.databaseid = dboid;
+ cached_tabent_key.objectid = reloid;
+
+ return &cached_tabent;
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2680,30 +3317,46 @@ pgstat_fetch_stat_tabentry(Oid relid)
*
* Support function for the SQL-callable pgstat* functions. Returns
* the collected statistics for one function or NULL.
+ *
+ * The returned entry is stored in static memory so the content is valid until
+ * the next call of the same function for the different function id.
* ----------
*/
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
-
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
-
- return funcentry;
+ PgStat_StatFuncEntry *shent;
+ Oid dboid = MyDatabaseId;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for the InvalidOid */
+ Assert(func_id != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_funcent_key.databaseid == dboid &&
+ cached_funcent_key.objectid == func_id)
+ return &cached_funcent;
+
+ shent = (PgStat_StatFuncEntry *)
+ get_stat_entry(PGSTAT_TYPE_FUNCTION, dboid, func_id, true, false,
+ NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_funcent, shent, sizeof(PgStat_StatFuncEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_funcent_key.databaseid = dboid;
+ cached_funcent_key.objectid = func_id;
+
+ return &cached_funcent;
}
-
/* ----------
* pgstat_fetch_stat_beentry() -
*
@@ -2763,53 +3416,160 @@ pgstat_fetch_stat_numbackends(void)
return localNumBackends;
}
+/*
+ * ---------
+ * pgstat_get_stat_timestamp() -
+ *
+ * Returns the last update timstamp of global staticstics.
+ */
+TimestampTz
+pgstat_get_stat_timestamp(void)
+{
+ return (TimestampTz) pg_atomic_read_u64(&StatsShmem->stats_timestamp);
+}
+
/*
* ---------
* pgstat_fetch_stat_archiver() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the archiver statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_ArchiverStats *
+PgStat_Archiver *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ PgStat_Archiver reset;
+ PgStat_Archiver *reset_shared = &StatsShmem->archiver_reset_offset;
+ PgStat_Archiver *shared = &StatsShmem->archiver_stats;
+ PgStat_Archiver *cached = &cached_archiverstats;
- return &archiverStats;
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_Archiver),
+ &StatsShmem->archiver_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_Archiver));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ if (cached->archived_count == reset.archived_count)
+ {
+ cached->last_archived_wal[0] = 0;
+ cached->last_archived_timestamp = 0;
+ }
+ cached->archived_count -= reset.archived_count;
+
+ if (cached->failed_count == reset.failed_count)
+ {
+ cached->last_failed_wal[0] = 0;
+ cached->last_failed_timestamp = 0;
+ }
+ cached->failed_count -= reset.failed_count;
+
+ cached->stat_reset_timestamp = reset.stat_reset_timestamp;
+
+ cached_archiverstats_is_valid = true;
+
+ return &cached_archiverstats;
}
/*
* ---------
- * pgstat_fetch_global() -
+ * pgstat_fetch_stat_bgwriter() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the global statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_GlobalStats *
-pgstat_fetch_global(void)
+PgStat_BgWriter *
+pgstat_fetch_stat_bgwriter(void)
{
- backend_read_statsfile();
+ PgStat_BgWriter reset;
+ PgStat_BgWriter *reset_shared = &StatsShmem->bgwriter_reset_offset;
+ PgStat_BgWriter *shared = &StatsShmem->bgwriter_stats;
+ PgStat_BgWriter *cached = &cached_bgwriterstats;
+
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_BgWriter),
+ &StatsShmem->bgwriter_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_BgWriter));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ cached->buf_written_clean -= reset.buf_written_clean;
+ cached->maxwritten_clean -= reset.maxwritten_clean;
+ cached->buf_alloc -= reset.buf_alloc;
+ cached->stat_reset_timestamp = reset.stat_reset_timestamp;
+
+ cached_bgwriterstats_is_valid = true;
+
+ return &cached_bgwriterstats;
+}
+
+/*
+ * ---------
+ * pgstat_fetch_stat_checkpinter() -
+ *
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
+ * ---------
+ */
+PgStat_CheckPointer *
+pgstat_fetch_stat_checkpointer(void)
+{
+ PgStat_CheckPointer reset;
+ PgStat_CheckPointer *reset_shared = &StatsShmem->checkpointer_reset_offset;
+ PgStat_CheckPointer *shared = &StatsShmem->checkpointer_stats;
+ PgStat_CheckPointer *cached = &cached_checkpointerstats;
+
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_CheckPointer),
+ &StatsShmem->checkpointer_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_CheckPointer));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ cached->timed_checkpoints -= reset.timed_checkpoints;
+ cached->requested_checkpoints -= reset.requested_checkpoints;
+ cached->buf_written_checkpoints -= reset.buf_written_checkpoints;
+ cached->buf_written_backend -= reset.buf_written_backend;
+ cached->buf_fsync_backend -= reset.buf_fsync_backend;
+ cached->checkpoint_write_time -= reset.checkpoint_write_time;
+ cached->checkpoint_sync_time -= reset.checkpoint_sync_time;
+
+ cached_checkpointerstats_is_valid = true;
- return &globalStats;
+ return &cached_checkpointerstats;
}
/*
* ---------
* pgstat_fetch_stat_wal() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the WAL statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned entry
+ * is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_WalStats *
+PgStat_Wal *
pgstat_fetch_stat_wal(void)
{
- backend_read_statsfile();
+ if (!cached_walstats_is_valid)
+ {
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&cached_walstats, &StatsShmem->wal_stats, sizeof(PgStat_Wal));
+ LWLockRelease(StatsLock);
+ }
- return &walStats;
+ cached_walstats_is_valid = true;
+
+ return &cached_walstats;
}
/*
@@ -2823,9 +3583,27 @@ pgstat_fetch_stat_wal(void)
PgStat_SLRUStats *
pgstat_fetch_slru(void)
{
- backend_read_statsfile();
+ size_t size = sizeof(PgStat_SLRUStats) * SLRU_NUM_ELEMENTS;
- return slruStats;
+ for (;;)
+ {
+ uint32 before_count;
+ uint32 after_count;
+
+ pg_read_barrier();
+ before_count = pg_atomic_read_u32(&StatsShmem->slru_changecount);
+ memcpy(&cached_slrustats, &StatsShmem->slru_stats, size);
+ after_count = pg_atomic_read_u32(&StatsShmem->slru_changecount);
+
+ if (before_count == after_count && (before_count & 1) == 0)
+ break;
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ cached_slrustats_is_valid = true;
+
+ return &cached_slrustats;
}
/*
@@ -2837,13 +3615,41 @@ pgstat_fetch_slru(void)
* number of entries in nslots_p.
* ---------
*/
-PgStat_ReplSlotStats *
+PgStat_ReplSlot *
pgstat_fetch_replslot(int *nslots_p)
{
- backend_read_statsfile();
- *nslots_p = nReplSlotStats;
- return replSlotStats;
+ if (cached_replslotstats == NULL)
+ {
+ cached_replslotstats = (PgStat_ReplSlot *)
+ MemoryContextAlloc(pgStatCacheContext,
+ sizeof(PgStat_ReplSlot) * max_replication_slots);
+ }
+
+ if (n_cached_replslotstats < 0)
+ {
+ int n = 0;
+ int i;
+
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ PgStat_ReplSlot *shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT, MyDatabaseId, i,
+ false, false, NULL);
+ if (shent && !shent->header.dropped)
+ {
+ memcpy(cached_replslotstats[n++].slotname,
+ shent->slotname,
+ sizeof(PgStat_ReplSlot) -
+ offsetof(PgStat_ReplSlot, slotname));
+ }
+ }
+
+ n_cached_replslotstats = n;
+ }
+
+ *nslots_p = n_cached_replslotstats;
+ return cached_replslotstats;
}
/* ------------------------------------------------------------
@@ -3068,8 +3874,8 @@ pgstat_initialize(void)
*/
prevWalUsage = pgWalUsage;
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3246,12 +4052,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3264,13 +4073,22 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
if (OidIsValid(MyDatabaseId))
pgstat_report_stat(true);
+ /*
+ * We need to clean up temporary slots before detaching shared statistics
+ * so that the statistics for temporary slots are properly removed.
+ */
+ if (MyReplicationSlot != NULL)
+ ReplicationSlotRelease();
+
+ ReplicationSlotCleanup();
+
/*
* Clear my status entry, following the protocol of bumping st_changecount
* before and after. We use a volatile pointer here to ensure the
@@ -3281,6 +4099,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3541,7 +4361,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3836,8 +4657,8 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
+ case WAIT_EVENT_READING_STATS_FILE:
+ event_name = "ReadingStatsFile";
break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
@@ -4491,94 +5312,80 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
+ * Report archiver statistics
* ----------
*/
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
+void
+pgstat_report_archiver(const char *xlog, bool failed)
{
- int rc;
+ TimestampTz now = GetCurrentTimestamp();
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
- ((PgStat_MsgHdr *) msg)->m_size = len;
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->archiver_changecount, 1);
+ Assert((before_count & 1) == 0);
- /* We'll retry after EINTR, but ignore all other failures */
- do
+ if (failed)
{
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
- * ----------
- */
-void
-pgstat_send_archiver(const char *xlog, bool failed)
-{
- PgStat_MsgArchiver msg;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- strlcpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ ++StatsShmem->archiver_stats.failed_count;
+ memcpy(&StatsShmem->archiver_stats.last_failed_wal, xlog,
+ sizeof(StatsShmem->archiver_stats.last_failed_wal));
+ StatsShmem->archiver_stats.last_failed_timestamp = now;
+ }
+ else
+ {
+ ++StatsShmem->archiver_stats.archived_count;
+ memcpy(&StatsShmem->archiver_stats.last_archived_wal, xlog,
+ sizeof(StatsShmem->archiver_stats.last_archived_wal));
+ StatsShmem->archiver_stats.last_archived_timestamp = now;
+ }
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->archiver_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+ PgStat_BgWriter *s = &StatsShmem->bgwriter_stats;
+ PgStat_BgWriter *l = &BgWriterStats;
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->bgwriter_changecount, 1);
+ Assert((before_count & 1) == 0);
+
+ s->buf_written_clean += l->buf_written_clean;
+ s->maxwritten_clean += l->maxwritten_clean;
+ s->buf_alloc += l->buf_alloc;
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->bgwriter_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4587,537 +5394,136 @@ pgstat_send_bgwriter(void)
}
/* ----------
- * pgstat_send_wal() -
+ * pgstat_report_checkpointer() -
*
- * Send WAL statistics to the collector
+ * Report checkpointer statistics
* ----------
*/
void
-pgstat_send_wal(void)
+pgstat_report_checkpointer(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgWal all_zeroes;
-
- WalUsage walusage;
-
- /*
- * Calculate how much WAL usage counters are increased by substracting the
- * previous counters from the current ones. Fill the results in WAL stats
- * message.
- */
- MemSet(&walusage, 0, sizeof(WalUsage));
- WalUsageAccumDiff(&walusage, &pgWalUsage, &prevWalUsage);
-
- WalStats.m_wal_records = walusage.wal_records;
- WalStats.m_wal_fpi = walusage.wal_fpi;
- WalStats.m_wal_bytes = walusage.wal_bytes;
+ static const PgStat_CheckPointer all_zeroes;
+ PgStat_CheckPointer *s = &StatsShmem->checkpointer_stats;
+ PgStat_CheckPointer *l = &CheckPointerStats;
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&WalStats, &all_zeroes, sizeof(PgStat_MsgWal)) == 0)
+ if (memcmp(&CheckPointerStats, &all_zeroes,
+ sizeof(PgStat_CheckPointer)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&WalStats.m_hdr, PGSTAT_MTYPE_WAL);
- pgstat_send(&WalStats, sizeof(WalStats));
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->checkpointer_changecount, 1);
+ Assert((before_count & 1) == 0);
- /*
- * Save the current counters for the subsequent calculation of WAL usage.
- */
- prevWalUsage = pgWalUsage;
+ s->timed_checkpoints += l->timed_checkpoints;
+ s->requested_checkpoints += l->requested_checkpoints;
+ s->checkpoint_write_time += l->checkpoint_write_time;
+ s->checkpoint_sync_time += l->checkpoint_sync_time;
+ s->buf_written_checkpoints += l->buf_written_checkpoints;
+ s->buf_written_backend += l->buf_written_backend;
+ s->buf_fsync_backend += l->buf_fsync_backend;
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->checkpointer_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
/*
* Clear out the statistics buffer, so it can be re-used.
*/
- MemSet(&WalStats, 0, sizeof(WalStats));
+ MemSet(&CheckPointerStats, 0, sizeof(CheckPointerStats));
}
/* ----------
- * pgstat_send_slru() -
+ * pgstat_report_wal() -
*
- * Send SLRU statistics to the collector
+ * Report WAL statistics
* ----------
*/
-static void
-pgstat_send_slru(void)
+void
+pgstat_report_wal(void)
{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgSLRU all_zeroes;
-
- for (int i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /*
- * This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
- */
- if (memcmp(&SLRUStats[i], &all_zeroes, sizeof(PgStat_MsgSLRU)) == 0)
- continue;
-
- /* set the SLRU type before each send */
- SLRUStats[i].m_index = i;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&SLRUStats[i].m_hdr, PGSTAT_MTYPE_SLRU);
- pgstat_send(&SLRUStats[i], sizeof(PgStat_MsgSLRU));
-
- /*
- * Clear out the statistics buffer, so it can be re-used.
- */
- MemSet(&SLRUStats[i], 0, sizeof(PgStat_MsgSLRU));
- }
+ flush_walstat(false);
}
-
/* ----------
- * PgstatCollectorMain() -
+ * get_local_dbstat_entry() -
*
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
- WaitEvent event;
- WaitEventSet *wes;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /* Prepare to wait for our latch or data in our socket. */
- wes = CreateWaitEventSet(CurrentMemoryContext, 3);
- AddWaitEventToSet(wes, WL_LATCH_SET, PGINVALID_SOCKET, MyLatch, NULL);
- AddWaitEventToSet(wes, WL_POSTMASTER_DEATH, PGINVALID_SOCKET, NULL, NULL);
- AddWaitEventToSet(wes, WL_SOCKET_READABLE, pgStatSock, NULL, NULL);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSLRUCOUNTER:
- pgstat_recv_resetslrucounter(&msg.msg_resetslrucounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETREPLSLOTCOUNTER:
- pgstat_recv_resetreplslotcounter(&msg.msg_resetreplslotcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_WAL:
- pgstat_recv_wal(&msg.msg_wal, len);
- break;
-
- case PGSTAT_MTYPE_SLRU:
- pgstat_recv_slru(&msg.msg_slru, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- case PGSTAT_MTYPE_REPLSLOT:
- pgstat_recv_replslot(&msg.msg_replslot, len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitEventSetWait(wes, -1L, &event, 1, WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitEventSetWait(wes, 2 * 1000L /* msec */ , &event, 1,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr == 1 && event.events == WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- FreeWaitEventSet(wes);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
*/
static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
+get_local_dbstat_entry(Oid dbid)
{
- PgStat_StatDBEntry *result;
+ PgStat_StatDBEntry *dbentry;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
/*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
+ * Find an entry or create a new one.
*/
- if (!found)
- reset_dbentry_counters(result);
+ dbentry = (PgStat_StatDBEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
- return result;
+ return dbentry;
}
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
*/
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
{
- PgStat_StatTabEntry *result;
+ PgStat_TableStatus *tabentry;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
+ tabentry = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->inserts_since_vacuum = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
+ return tabentry;
}
-
/* ----------
- * pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
+ * pgstat_write_statsfile() -
+ * Write the global statistics file, as well as DB files.
*
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * This function is called in the last process that is accessing the shared
+ * stats so locking is not required.
* ----------
*/
static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+pgstat_write_statsfile(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
- int i;
+ dshash_seq_status hstat;
+ PgStatHashEntry *ps;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
+
+ /* this is the last process that is accesing the shared stats */
+#ifdef USE_ASSERT_CHECKING
+ LWLockAcquire(StatsLock, LW_SHARED);
+ Assert(StatsShmem->refcount == 0);
+ LWLockRelease(StatsLock);
+#endif
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -5137,7 +5543,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ pg_atomic_write_u64(&StatsShmem->stats_timestamp, GetCurrentTimestamp());
/*
* Write the file header --- currently just a format ID.
@@ -5147,200 +5553,72 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
/*
- * Write global stats struct
+ * Write bgwriter global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(&StatsShmem->bgwriter_stats, sizeof(PgStat_BgWriter), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Write archiver stats struct
+ * Write checkpointer global stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(&StatsShmem->checkpointer_stats, sizeof(PgStat_CheckPointer), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Write WAL stats struct
+ * Write archiver global stats struct
*/
- rc = fwrite(&walStats, sizeof(walStats), 1, fpout);
+ rc = fwrite(&StatsShmem->archiver_stats, sizeof(PgStat_Archiver), 1,
+ fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Write WAL global stats struct
+ */
+ rc = fwrite(&StatsShmem->wal_stats, sizeof(PgStat_Wal), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write SLRU stats struct
*/
- rc = fwrite(slruStats, sizeof(slruStats), 1, fpout);
+ rc = fwrite(&StatsShmem->slru_stats, sizeof(PgStatSharedSLRUStats), 1,
+ fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Walk through the database table.
+ * Walk through the stats entry
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((ps = dshash_seq_next(&hstat)) != NULL)
{
- /*
- * Write out the table and function stats for this DB into the
- * appropriate per-DB stat file, if required.
- */
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
+ PgStat_StatEntryHeader *shent;
+ size_t len;
+
+ CHECK_FOR_INTERRUPTS();
+
+ shent = (PgStat_StatEntryHeader *)dsa_get_address(area, ps->body);
+
+ /* we may have some "dropped" entries not yet removed, skip them */
+ if (shent->dropped)
+ continue;
+
+ /* Make DB's timestamp consistent with the global stats */
+ if (ps->key.type == PGSTAT_TYPE_DB)
{
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) shent;
- pgstat_write_db_statsfile(dbentry, permanent);
+ dbentry->stats_timestamp =
+ (TimestampTz) pg_atomic_read_u64(&StatsShmem->stats_timestamp);
}
- /*
- * Write out the DB entry. We don't write the tables or functions
- * pointers, since they're of no use to any other process.
- */
- fputc('D', fpout);
- rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * Write replication slot stats struct
- */
- for (i = 0; i < nReplSlotStats; i++)
- {
- fputc('R', fpout);
- rc = fwrite(&replSlotStats[i], sizeof(PgStat_ReplSlotStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
- */
- fputc('E', fpout);
-
- if (ferror(fpout))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write temporary statistics file \"%s\": %m",
- tmpfile)));
- FreeFile(fpout);
- unlink(tmpfile);
- }
- else if (FreeFile(fpout) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not close temporary statistics file \"%s\": %m",
- tmpfile)));
- unlink(tmpfile);
- }
- else if (rename(tmpfile, statfile) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
- tmpfile, statfile)));
- unlink(tmpfile);
- }
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
-}
-
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
-
-/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
- * ----------
- */
-static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
-{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpout;
- int32 format_id;
- Oid dbid = dbentry->databaseid;
- int rc;
- char tmpfile[MAXPGPATH];
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "writing stats file \"%s\"", statfile);
-
- /*
- * Open the statistics temp file to write out the current values.
- */
- fpout = AllocateFile(tmpfile, PG_BINARY_W);
- if (fpout == NULL)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not open temporary statistics file \"%s\": %m",
- tmpfile)));
- return;
- }
-
- /*
- * Write the file header --- currently just a format ID.
- */
- format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Walk through the database's access stats per table.
- */
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
- {
- fputc('T', fpout);
- rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
+ fputc('S', fpout);
+ rc = fwrite(&ps->key, sizeof(PgStatHashKey), 1, fpout);
- /*
- * Walk through the database's function stats table.
- */
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
- {
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ /* Write except the header part of the etnry */
+ len = PGSTAT_SHENT_BODY_LEN(ps->key.type);
+ rc = fwrite(PGSTAT_SHENT_BODY(shent), len, 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ dshash_seq_term(&hstat);
/*
* No more output to be done. Close the temp file and replace the old
@@ -5374,113 +5652,63 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
}
/* ----------
- * pgstat_read_statsfiles() -
- *
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * pgstat_read_statsfile() -
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
+ * Reads in existing activity statistics file into the shared stats hash.
*
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
+ * This function is called in the only process that is accessing the shared
+ * stats so locking is not required.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+static void
+pgstat_read_statsfile(void)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
- int i;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+ char tag;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ /* this is the only process that is accesing the shared stats */
+#ifdef USE_ASSERT_CHECKING
+ LWLockAcquire(StatsLock, LW_SHARED);
+ Assert(StatsShmem->refcount == 1);
+ LWLockRelease(StatsLock);
+#endif
- /* Allocate the space for replication slot statistics */
- replSlotStats = palloc0(max_replication_slots * sizeof(PgStat_ReplSlotStats));
- nReplSlotStats = 0;
-
- /*
- * Clear out global, archiver, WAL and SLRU statistics so they start from
- * zero in case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
- memset(&walStats, 0, sizeof(walStats));
- memset(&slruStats, 0, sizeof(slruStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
- walStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all SLRU items too.
- */
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- slruStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all replication slots too.
- */
- for (i = 0; i < max_replication_slots; i++)
- replSlotStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ StatsShmem->bgwriter_stats.stat_reset_timestamp = GetCurrentTimestamp();
+ StatsShmem->archiver_stats.stat_reset_timestamp =
+ StatsShmem->bgwriter_stats.stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5489,681 +5717,150 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- goto done;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
- goto done;
- }
-
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
- /*
- * Read archiver stats struct
- */
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
- goto done;
- }
-
- /*
- * Read WAL stats struct
- */
- if (fread(&walStats, 1, sizeof(walStats), fpin) != sizeof(walStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&walStats, 0, sizeof(walStats));
goto done;
}
/*
- * Read SLRU stats struct
+ * Read bgwiter stats struct
*/
- if (fread(slruStats, 1, sizeof(slruStats), fpin) != sizeof(slruStats))
+ if (fread(&StatsShmem->bgwriter_stats, 1, sizeof(PgStat_BgWriter), fpin) !=
+ sizeof(PgStat_BgWriter))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&slruStats, 0, sizeof(slruStats));
+ MemSet(&StatsShmem->bgwriter_stats, 0, sizeof(PgStat_BgWriter));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Add to the DB hash
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
-
- break;
-
- /*
- * 'R' A PgStat_ReplSlotStats struct describing a replication
- * slot follows.
- */
- case 'R':
- if (fread(&replSlotStats[nReplSlotStats], 1, sizeof(PgStat_ReplSlotStats), fpin)
- != sizeof(PgStat_ReplSlotStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- memset(&replSlotStats[nReplSlotStats], 0, sizeof(PgStat_ReplSlotStats));
- goto done;
- }
- nReplSlotStats++;
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-
- return dbhash;
-}
-
-
-/* ----------
- * pgstat_read_db_statsfile() -
- *
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
- * ----------
- */
-static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
-{
- PgStat_StatTabEntry *tabentry;
- PgStat_StatTabEntry tabbuf;
- PgStat_StatFuncEntry funcbuf;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpin;
- int32 format_id;
- bool found;
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
-
- /*
- * Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
- *
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
- * condition is suspicious.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return;
- }
-
- /*
- * Verify it's of the expected format.
+ * Read checkpointer stats struct
*/
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
+ if (fread(&StatsShmem->checkpointer_stats, 1, sizeof(PgStat_CheckPointer), fpin) !=
+ sizeof(PgStat_CheckPointer))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
+ MemSet(&StatsShmem->checkpointer_stats, 0, sizeof(PgStat_CheckPointer));
goto done;
}
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'T' A PgStat_StatTabEntry follows.
- */
- case 'T':
- if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
- fpin) != sizeof(PgStat_StatTabEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(tabentry, &tabbuf, sizeof(tabbuf));
- break;
-
- /*
- * 'F' A PgStat_StatFuncEntry follows.
- */
- case 'F':
- if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
- fpin) != sizeof(PgStat_StatFuncEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(funcentry, &funcbuf, sizeof(funcbuf));
- break;
-
- /*
- * 'E' The EOF marker of a complete stats file.
- */
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts. The caller must
- * rely on timestamp stored in *ts iff the function returns true.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- PgStat_WalStats myWalStats;
- PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
- PgStat_ReplSlotStats myReplSlotStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
/*
* Read archiver stats struct
*/
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
+ if (fread(&StatsShmem->archiver_stats, 1, sizeof(PgStat_Archiver),
+ fpin) != sizeof(PgStat_Archiver))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
+ MemSet(&StatsShmem->archiver_stats, 0, sizeof(PgStat_Archiver));
+ goto done;
}
/*
* Read WAL stats struct
*/
- if (fread(&myWalStats, 1, sizeof(myWalStats), fpin) != sizeof(myWalStats))
+ if (fread(&StatsShmem->wal_stats, 1, sizeof(PgStat_Wal), fpin)
+ != sizeof(PgStat_Wal))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
+ MemSet(&StatsShmem->wal_stats, 0, sizeof(PgStat_Wal));
+ goto done;
}
/*
* Read SLRU stats struct
*/
- if (fread(mySLRUStats, 1, sizeof(mySLRUStats), fpin) != sizeof(mySLRUStats))
+ if (fread(&StatsShmem->slru_stats, 1, sizeof(PgStatSharedSLRUStats),
+ fpin) != sizeof(PgStatSharedSLRUStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- /*
- * 'R' A PgStat_ReplSlotStats struct describing a replication
- * slot follows.
- */
- case 'R':
- if (fread(&myReplSlotStats, 1, sizeof(PgStat_ReplSlotStats), fpin)
- != sizeof(PgStat_ReplSlotStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
- break;
-
- case 'E':
- goto done;
-
- default:
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
- }
+ goto done;
}
-done:
- FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
/*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
+ while ((tag = fgetc(fpin)) == 'S')
{
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
+ PgStatHashKey key;
+ PgStat_StatEntryHeader *p;
+ size_t len;
CHECK_FOR_INTERRUPTS();
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
+ if (fread(&key, 1, sizeof(key), fpin) != sizeof(key))
{
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
}
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- ereport(LOG,
- (errmsg("statistics collector's time %s is later than backend local time %s",
- filetime, mytime)));
- pfree(filetime);
- pfree(mytime);
- }
+ p = get_stat_entry(key.type, key.databaseid, key.objectid,
+ false, true, &found);
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
+ /* don't allow duplicate entries */
+ if (found)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
}
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
+ /* Avoid overwriting header part */
+ len = PGSTAT_SHENT_BODY_LEN(key.type);
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
+ if (fread(PGSTAT_SHENT_BODY(p), 1, len, fpin) != len)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
}
- if (count >= PGSTAT_POLL_LOOP_COUNT)
+ if (tag != 'E')
+ {
ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
+done:
+ FreeFile(fpin);
+
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
+
+ return;
}
-
/* ----------
* pgstat_setup_memcxt() -
*
- * Create pgStatLocalContext, if not already done.
+ * Create pgStatLocalContext if not already done.
* ----------
*/
static void
pgstat_setup_memcxt(void)
{
if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
-}
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+ if (!pgStatCacheContext)
+ pgStatCacheContext =
+ AllocSetContextCreate(CacheMemoryContext,
+ "Activity statistics",
+ ALLOCSET_SMALL_SIZES);
+}
/* ----------
* pgstat_clear_snapshot() -
@@ -6180,906 +5877,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- ereport(LOG,
- (errmsg("stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid)));
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum = tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- tabentry->inserts_since_vacuum = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum += tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_WAL)
- {
- /* Reset the WAL statistics for the cluster. */
- memset(&walStats, 0, sizeof(walStats));
- walStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_resetslrucounter() -
- *
- * Reset some SLRU statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len)
-{
- int i;
- TimestampTz ts = GetCurrentTimestamp();
-
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /* reset entry with the given index, or all entries (index is -1) */
- if ((msg->m_index == -1) || (msg->m_index == i))
- {
- memset(&slruStats[i], 0, sizeof(slruStats[i]));
- slruStats[i].stat_reset_timestamp = ts;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_resetreplslotcounter() -
- *
- * Reset some replication slot statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
- int len)
-{
- int i;
- int idx = -1;
- TimestampTz ts;
-
- ts = GetCurrentTimestamp();
- if (msg->clearall)
- {
- for (i = 0; i < nReplSlotStats; i++)
- pgstat_reset_replslot(i, ts);
- }
- else
- {
- /* Get the index of replication slot statistics to reset */
- idx = pgstat_replslot_index(msg->m_slotname, false);
-
- /*
- * Nothing to do if the given slot entry is not found. This could
- * happen when the slot with the given name is removed and the
- * corresponding statistics entry is also removed before receiving the
- * reset message.
- */
- if (idx < 0)
- return;
-
- /* Reset the stats for the requested replication slot */
- pgstat_reset_replslot(idx, ts);
- }
-}
-
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signaling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * It is quite possible that a non-aggressive VACUUM ended up skipping
- * various pages, however, we'll zero the insert counter here regardless.
- * It's currently used only to track when we need to perform an "insert"
- * autovacuum, which are mainly intended to freeze newly inserted tuples.
- * Zeroing this may just mean we'll not try to vacuum the table again
- * until enough tuples have been inserted to trigger another insert
- * autovacuum. An anti-wraparound autovacuum will catch any persistent
- * stragglers.
- */
- tabentry->inserts_since_vacuum = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_wal() -
- *
- * Process a WAL message.
- * ----------
- */
-static void
-pgstat_recv_wal(PgStat_MsgWal *msg, int len)
-{
- walStats.wal_records += msg->m_wal_records;
- walStats.wal_fpi += msg->m_wal_fpi;
- walStats.wal_bytes += msg->m_wal_bytes;
- walStats.wal_buffers_full += msg->m_wal_buffers_full;
-}
-
-/* ----------
- * pgstat_recv_slru() -
- *
- * Process a SLRU message.
- * ----------
- */
-static void
-pgstat_recv_slru(PgStat_MsgSLRU *msg, int len)
-{
- slruStats[msg->m_index].blocks_zeroed += msg->m_blocks_zeroed;
- slruStats[msg->m_index].blocks_hit += msg->m_blocks_hit;
- slruStats[msg->m_index].blocks_read += msg->m_blocks_read;
- slruStats[msg->m_index].blocks_written += msg->m_blocks_written;
- slruStats[msg->m_index].blocks_exists += msg->m_blocks_exists;
- slruStats[msg->m_index].flush += msg->m_flush;
- slruStats[msg->m_index].truncate += msg->m_truncate;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_replslot() -
- *
- * Process a REPLSLOT message.
- * ----------
- */
-static void
-pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len)
-{
- int idx;
-
- /*
- * Get the index of replication slot statistics. On dropping, we don't
- * create the new statistics.
- */
- idx = pgstat_replslot_index(msg->m_slotname, !msg->m_drop);
-
- /*
- * The slot entry is not found or there is no space to accommodate the new
- * entry. This could happen when the message for the creation of a slot
- * reached before the drop message even though the actual operations
- * happen in reverse order. In such a case, the next update of the
- * statistics for the same slot will create the required entry.
- */
- if (idx < 0)
- return;
-
- /* it must be a valid replication slot index */
- Assert(idx < nReplSlotStats);
-
- if (msg->m_drop)
- {
- /* Remove the replication slot statistics with the given name */
- if (idx < nReplSlotStats - 1)
- memcpy(&replSlotStats[idx], &replSlotStats[nReplSlotStats - 1],
- sizeof(PgStat_ReplSlotStats));
- nReplSlotStats--;
- }
- else
- {
- /* Update the replication slot statistics */
- replSlotStats[idx].spill_txns += msg->m_spill_txns;
- replSlotStats[idx].spill_count += msg->m_spill_count;
- replSlotStats[idx].spill_bytes += msg->m_spill_bytes;
- replSlotStats[idx].stream_txns += msg->m_stream_txns;
- replSlotStats[idx].stream_count += msg->m_stream_count;
- replSlotStats[idx].stream_bytes += msg->m_stream_bytes;
- }
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
+ }
+
+ /* Invalidate the simple cache keys */
+ cached_dbent_key = stathashkey_zero;
+ cached_tabent_key = stathashkey_zero;
+ cached_funcent_key = stathashkey_zero;
+ cached_archiverstats_is_valid = false;
+ cached_bgwriterstats_is_valid = false;
+ cached_checkpointerstats_is_valid = false;
+ cached_walstats_is_valid = false;
+ cached_slrustats_is_valid = false;
+ n_cached_replslotstats = -1;
}
/*
@@ -7126,60 +5942,6 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
-/* ----------
- * pgstat_replslot_index
- *
- * Return the index of entry of a replication slot with the given name, or
- * -1 if the slot is not found.
- *
- * create_it tells whether to create the new slot entry if it is not found.
- * ----------
- */
-static int
-pgstat_replslot_index(const char *name, bool create_it)
-{
- int i;
-
- Assert(nReplSlotStats <= max_replication_slots);
- for (i = 0; i < nReplSlotStats; i++)
- {
- if (strcmp(replSlotStats[i].slotname, name) == 0)
- return i; /* found */
- }
-
- /*
- * The slot is not found. We don't want to register the new statistics if
- * the list is already full or the caller didn't request.
- */
- if (i == max_replication_slots || !create_it)
- return -1;
-
- /* Register new slot */
- memset(&replSlotStats[nReplSlotStats], 0, sizeof(PgStat_ReplSlotStats));
- strlcpy(replSlotStats[nReplSlotStats].slotname, name, NAMEDATALEN);
-
- return nReplSlotStats++;
-}
-
-/* ----------
- * pgstat_reset_replslot
- *
- * Reset the replication slot stats at index 'i'.
- * ----------
- */
-static void
-pgstat_reset_replslot(int i, TimestampTz ts)
-{
- /* reset only counters. Don't clear slot name */
- replSlotStats[i].spill_txns = 0;
- replSlotStats[i].spill_count = 0;
- replSlotStats[i].spill_bytes = 0;
- replSlotStats[i].stream_txns = 0;
- replSlotStats[i].stream_count = 0;
- replSlotStats[i].stream_bytes = 0;
- replSlotStats[i].stat_reset_timestamp = ts;
-}
-
/*
* pgstat_slru_index
*
@@ -7224,7 +5986,7 @@ pgstat_slru_name(int slru_idx)
* Returns pointer to entry with counters for given SLRU (based on the name
* stored in SlruCtl as lwlock tranche name).
*/
-static inline PgStat_MsgSLRU *
+static PgStat_SLRUStats *
slru_entry(int slru_idx)
{
/*
@@ -7235,7 +5997,7 @@ slru_entry(int slru_idx)
Assert((slru_idx >= 0) && (slru_idx < SLRU_NUM_ELEMENTS));
- return &SLRUStats[slru_idx];
+ return &local_SLRUStats[slru_idx];
}
/*
@@ -7245,41 +6007,48 @@ slru_entry(int slru_idx)
void
pgstat_count_slru_page_zeroed(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_zeroed += 1;
+ slru_entry(slru_idx)->blocks_zeroed += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_hit(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_hit += 1;
+ slru_entry(slru_idx)->blocks_hit += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_exists(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_exists += 1;
+ slru_entry(slru_idx)->blocks_exists += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_read(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_read += 1;
+ slru_entry(slru_idx)->blocks_read += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_written(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_written += 1;
+ slru_entry(slru_idx)->blocks_written += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_flush(int slru_idx)
{
- slru_entry(slru_idx)->m_flush += 1;
+ slru_entry(slru_idx)->flush += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_truncate(int slru_idx)
{
- slru_entry(slru_idx)->m_truncate += 1;
+ slru_entry(slru_idx)->truncate += 1;
+ have_slrustats = true;
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index af91c313e2..9b9c9b1c11 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -251,7 +251,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -512,7 +511,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1327,12 +1325,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1782,11 +1774,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2699,8 +2686,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3029,8 +3014,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3097,13 +3080,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3176,22 +3152,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3653,22 +3613,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3895,8 +3839,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3920,8 +3862,7 @@ PostmasterStateMachine(void)
{
/*
* PM_WAIT_DEAD_END state ends when the BackendList is entirely empty
- * (ie, no dead_end children remain), and the archiver and stats
- * collector are gone too.
+ * (ie, no dead_end children remain), and the archiveris gone too.
*
* The reason we wait for those two is to protect them against a new
* postmaster starting conflicting subprocesses; this isn't an
@@ -3931,8 +3872,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4133,8 +4073,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5067,18 +5005,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5191,12 +5117,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6113,7 +6033,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6169,8 +6088,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6403,7 +6320,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 0f54635550..d21801cf90 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1568,8 +1568,8 @@ is_checksummed_file(const char *fullpath, const char *filename)
*
* If 'missing_ok' is true, will not throw an error if the file is not found.
*
- * If dboid is anything other than InvalidOid then any checksum failures detected
- * will get reported to the stats collector.
+ * If dboid is anything other than InvalidOid then any checksum failures
+ * detected will get reported to the activity stats facility.
*
* Returns true if the file was successfully sent, false if 'missing_ok',
* and the file did not exist.
diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c
index e00c7ffc01..608818beea 100644
--- a/src/backend/replication/slot.c
+++ b/src/backend/replication/slot.c
@@ -692,14 +692,10 @@ ReplicationSlotDropPtr(ReplicationSlot *slot)
(errmsg("could not remove directory \"%s\"", tmppath)));
/*
- * Send a message to drop the replication slot to the stats collector.
- * Since there is no guarantee of the order of message transfer on a UDP
- * connection, it's possible that a message for creating a new slot
- * reaches before a message for removing the old slot. We send the drop
- * and create messages while holding ReplicationSlotAllocationLock to
- * reduce that possibility. If the messages reached in reverse, we would
- * lose one statistics update message. But the next update message will
- * create the statistics for the replication slot.
+ * Drop the statistics entry for the replication slot. Do this while
+ * holding ReplicationSlotAllocationLock so that we don't drop a statistics
+ * entry for another slot with the same name just created on another
+ * session.
*/
if (SlotIsLogical(slot))
pgstat_report_replslot_drop(NameStr(slot->data.name));
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 561c212092..517354bed2 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2059,7 +2059,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ CheckPointerStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2169,7 +2169,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2359,7 +2359,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2367,7 +2367,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index f9bbe97b50..78cfe91eab 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -149,6 +149,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -267,6 +268,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index db7e59f8b7..e6e4c0fb04 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -177,7 +177,9 @@ static const char *const BuiltinTrancheNames[] = {
/* LWTRANCHE_PARALLEL_APPEND: */
"ParallelAppend",
/* LWTRANCHE_PER_XACT_PREDICATE_LIST: */
- "PerXactPredicateList"
+ "PerXactPredicateList",
+ /* LWTRANCHE_STATS: */
+ "ActivityStatistics"
};
StaticAssertDecl(lengthof(BuiltinTrancheNames) ==
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index 774292fd94..91bf8d5b5d 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -53,3 +53,4 @@ XactTruncationLock 44
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+StatsLock 48
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 4dc24649df..75d1695576 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -414,8 +414,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
}
/*
- * It'd be nice to tell the stats collector to forget them immediately,
- * too. But we can't because we don't know the OIDs.
+ * It'd be nice to tell the activity stats facility to forget them
+ * immediately, too. But we can't because we don't know the OIDs.
*/
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 28055680aa..8b51852505 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3271,6 +3271,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3842,6 +3848,7 @@ PostgresMain(int argc, char *argv[],
volatile bool send_ready_for_query = true;
bool idle_in_transaction_timeout_enabled = false;
bool idle_session_timeout_enabled = false;
+ bool idle_stats_update_timeout_enabled = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4238,11 +4245,12 @@ PostgresMain(int argc, char *argv[],
* Note: this includes fflush()'ing the last of the prior output.
*
* This is also a good time to send collected statistics to the
- * collector, and to update the PS stats display. We avoid doing
- * those every time through the message loop because it'd slow down
- * processing of batched messages, and because we don't want to report
- * uncommitted updates (that confuses autovacuum). The notification
- * processor wants a call too, if we are not in a transaction block.
+ * activity statistics, and to update the PS stats display. We avoid
+ * doing those every time through the message loop because it'd slow
+ * down processing of batched messages, and because we don't want to
+ * report uncommitted updates (that confuses autovacuum). The
+ * notification processor wants a call too, if we are not in a
+ * transaction block.
*
* Also, if an idle timeout is enabled, start the timer for that.
*/
@@ -4276,6 +4284,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4288,8 +4298,14 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ /* Start the idle-stats-update timer */
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ idle_stats_update_timeout_enabled = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
@@ -4323,9 +4339,9 @@ PostgresMain(int argc, char *argv[],
firstchar = ReadCommand(&input_message);
/*
- * (4) turn off the idle-in-transaction and idle-session timeouts, if
- * active. We do this before step (5) so that any last-moment timeout
- * is certain to be detected in step (5).
+ * (4) turn off the idle-in-transaction, idle-session and
+ * idle-state-update timeouts if active. We do this before step (5) so
+ * that any last-moment timeout is certain to be detected in step (5).
*
* At most one of these timeouts will be active, so there's no need to
* worry about combining the timeout.c calls into one.
@@ -4340,6 +4356,11 @@ PostgresMain(int argc, char *argv[],
disable_timeout(IDLE_SESSION_TIMEOUT, false);
idle_session_timeout_enabled = false;
}
+ if (idle_stats_update_timeout_enabled)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ idle_stats_update_timeout_enabled = false;
+ }
/*
* (5) disable async signal conditions again.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5c12a165a1..fd6465dfb9 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1,7 +1,7 @@
/*-------------------------------------------------------------------------
*
* pgstatfuncs.c
- * Functions for accessing the statistics collector data
+ * Functions for accessing the activity statistics data
*
* Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -35,9 +35,6 @@
#define HAS_PGSTAT_PERMISSIONS(role) (is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS) || has_privs_of_role(GetUserId(), role))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1269,7 +1266,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1285,7 +1282,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1301,7 +1298,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1317,7 +1314,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1333,7 +1330,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1349,7 +1346,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1365,7 +1362,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1381,7 +1378,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1397,7 +1394,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1430,7 +1427,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1446,7 +1443,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1461,7 +1458,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1476,7 +1473,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1491,7 +1488,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1506,7 +1503,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1521,7 +1518,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1536,11 +1533,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1555,7 +1552,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1573,7 +1570,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1610,7 +1607,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1626,7 +1623,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1634,69 +1631,71 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
Datum
pg_stat_get_bgwriter_timed_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->timed_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->timed_checkpoints);
}
Datum
pg_stat_get_bgwriter_requested_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->requested_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->requested_checkpoints);
}
Datum
pg_stat_get_bgwriter_buf_written_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_checkpoints);
}
Datum
pg_stat_get_bgwriter_buf_written_clean(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_clean);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_written_clean);
}
Datum
pg_stat_get_bgwriter_maxwritten_clean(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->maxwritten_clean);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->maxwritten_clean);
}
Datum
pg_stat_get_checkpoint_write_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
- PG_RETURN_FLOAT8((double) pgstat_fetch_global()->checkpoint_write_time);
+ PG_RETURN_FLOAT8((double)
+ pgstat_fetch_stat_checkpointer()->checkpoint_write_time);
}
Datum
pg_stat_get_checkpoint_sync_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
- PG_RETURN_FLOAT8((double) pgstat_fetch_global()->checkpoint_sync_time);
+ PG_RETURN_FLOAT8((double)
+ pgstat_fetch_stat_checkpointer()->checkpoint_sync_time);
}
Datum
pg_stat_get_bgwriter_stat_reset_time(PG_FUNCTION_ARGS)
{
- PG_RETURN_TIMESTAMPTZ(pgstat_fetch_global()->stat_reset_timestamp);
+ PG_RETURN_TIMESTAMPTZ(pgstat_fetch_stat_bgwriter()->stat_reset_timestamp);
}
Datum
pg_stat_get_buf_written_backend(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_backend);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_backend);
}
Datum
pg_stat_get_buf_fsync_backend(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_fsync_backend);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_fsync_backend);
}
Datum
pg_stat_get_buf_alloc(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_alloc);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_alloc);
}
/*
@@ -1710,7 +1709,7 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_WAL_COLS];
bool nulls[PG_STAT_GET_WAL_COLS];
char buf[256];
- PgStat_WalStats *wal_stats;
+ PgStat_Wal *wal_stats;
/* Initialise values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
@@ -1735,11 +1734,11 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
wal_stats = pgstat_fetch_stat_wal();
/* Fill values and NULLs */
- values[0] = Int64GetDatum(wal_stats->wal_records);
- values[1] = Int64GetDatum(wal_stats->wal_fpi);
+ values[0] = Int64GetDatum(wal_stats->wal_usage.wal_records);
+ values[1] = Int64GetDatum(wal_stats->wal_usage.wal_fpi);
/* Convert to numeric. */
- snprintf(buf, sizeof buf, UINT64_FORMAT, wal_stats->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, wal_stats->wal_usage.wal_bytes);
values[2] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
@@ -2020,7 +2019,7 @@ pg_stat_get_xact_function_self_time(PG_FUNCTION_ARGS)
Datum
pg_stat_get_snapshot_timestamp(PG_FUNCTION_ARGS)
{
- PG_RETURN_TIMESTAMPTZ(pgstat_fetch_global()->stats_timestamp);
+ PG_RETURN_TIMESTAMPTZ(pgstat_get_stat_timestamp());
}
/* Discard the active statistics snapshot */
@@ -2108,7 +2107,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDesc tupdesc;
Datum values[7];
bool nulls[7];
- PgStat_ArchiverStats *archiver_stats;
+ PgStat_Archiver *archiver_stats;
/* Initialise values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
@@ -2178,7 +2177,7 @@ pg_stat_get_replication_slots(PG_FUNCTION_ARGS)
Tuplestorestate *tupstore;
MemoryContext per_query_ctx;
MemoryContext oldcontext;
- PgStat_ReplSlotStats *slotstats;
+ PgStat_ReplSlot *slotstats;
int nstats;
int i;
@@ -2211,7 +2210,7 @@ pg_stat_get_replication_slots(PG_FUNCTION_ARGS)
{
Datum values[PG_STAT_GET_REPLICATION_SLOT_COLS];
bool nulls[PG_STAT_GET_REPLICATION_SLOT_COLS];
- PgStat_ReplSlotStats *s = &(slotstats[i]);
+ PgStat_ReplSlot *s = &(slotstats[i]);
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..0762c2034c 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -72,6 +72,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
+#include "pgstat.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rowsecurity.h"
#include "storage/lmgr.h"
@@ -2366,6 +2367,10 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
*/
RelationCloseSmgr(relation);
+ /* break mutual link with stats entry */
+ pgstat_delinkstats(relation);
+
+ if (relation->rd_rel)
/*
* Free all the subsidiary data structures of the relcache entry, then the
* entry itself.
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index ea28769d6a..997afcab6d 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -34,6 +34,7 @@ volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 0f67b99cc5..2567668b6c 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -269,9 +269,6 @@ GetBackendTypeDesc(BackendType backendType)
case B_ARCHIVER:
backendDesc = "archiver";
break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
case B_LOGGER:
backendDesc = "logger";
break;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e5965bc517..d4c17fd7ab 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
static void IdleSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -621,6 +622,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
RegisterTimeout(IDLE_SESSION_TIMEOUT, IdleSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1243,6 +1246,14 @@ IdleSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 17579eeaca..85299e2138 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -746,8 +746,8 @@ const char *const config_group_names[] =
gettext_noop("Statistics"),
/* STATS_MONITORING */
gettext_noop("Statistics / Monitoring"),
- /* STATS_COLLECTOR */
- gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* STATS_ACTIVITY */
+ gettext_noop("Statistics / Query and Index Activity Statistics"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
@@ -1457,7 +1457,7 @@ static struct config_bool ConfigureNamesBool[] =
#endif
{
- {"track_activities", PGC_SUSET, STATS_COLLECTOR,
+ {"track_activities", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects information about executing commands."),
gettext_noop("Enables the collection of information on the currently "
"executing command of each session, along with "
@@ -1468,7 +1468,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_counts", PGC_SUSET, STATS_COLLECTOR,
+ {"track_counts", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects statistics on database activity."),
NULL
},
@@ -1477,7 +1477,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_io_timing", PGC_SUSET, STATS_COLLECTOR,
+ {"track_io_timing", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects timing statistics for database I/O activity."),
NULL
},
@@ -4356,7 +4356,7 @@ static struct config_string ConfigureNamesString[] =
},
{
- {"stats_temp_directory", PGC_SIGHUP, STATS_COLLECTOR,
+ {"stats_temp_directory", PGC_SIGHUP, STATS_ACTIVITY,
gettext_noop("Writes temporary statistics files to the specified directory."),
NULL,
GUC_SUPERUSER_ONLY
@@ -4692,7 +4692,7 @@ static struct config_enum ConfigureNamesEnum[] =
},
{
- {"track_functions", PGC_SUSET, STATS_COLLECTOR,
+ {"track_functions", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects function-level statistics on database activity."),
NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 8930a94fff..4f5b6bdb12 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -580,7 +580,7 @@
# STATISTICS
#------------------------------------------------------------------------------
-# - Query and Index Statistics Collector -
+# - Query and Index Activity Statistics -
#track_activities = on
#track_counts = on
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index f674a7c94e..c61b828bf3 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 109;
+use Test::More tests => 108;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -124,7 +124,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index adb9f819bb..12708b9470 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -84,6 +84,8 @@ extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -322,7 +324,6 @@ typedef enum BackendType
B_WAL_SENDER,
B_WAL_WRITER,
B_ARCHIVER,
- B_STATS_COLLECTOR,
B_LOGGER,
} BackendType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c38b689710..0472b728bf 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2021, PostgreSQL Global Development Group
*
@@ -12,12 +12,15 @@
#define PGSTAT_H
#include "datatype/timestamp.h"
+#include "executor/instrument.h"
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -27,8 +30,8 @@
* ----------
*/
#define PGSTAT_STAT_PERMANENT_DIRECTORY "pg_stat"
-#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
-#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
+#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/saved_stats"
+#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/saved_stats.tmp"
/* Default directory to store temporary statistics data in */
#define PG_STAT_TMP_DIR "pg_stat_tmp"
@@ -41,38 +44,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_RESETSLRUCOUNTER,
- PGSTAT_MTYPE_RESETREPLSLOTCOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_WAL,
- PGSTAT_MTYPE_SLRU,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE,
- PGSTAT_MTYPE_REPLSLOT,
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -83,9 +54,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -159,10 +129,13 @@ typedef enum PgStat_Single_Reset_Type
*/
typedef struct PgStat_TableStatus
{
- Oid t_id; /* table's OID */
- bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
+ Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
/* ----------
@@ -186,353 +159,57 @@ typedef struct PgStat_TableXactStatus
struct PgStat_TableXactStatus *next; /* next of same subxact */
} PgStat_TableXactStatus;
-
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
+/*
+ * Archiver statistics kept in the shared stats
*/
-typedef struct PgStat_MsgHdr
+typedef struct PgStat_Archiver
{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
+ PgStat_Counter archived_count; /* archival successes */
+ char last_archived_wal[MAX_XFN_CHARS + 1]; /* last WAL file
+ * archived */
+ TimestampTz last_archived_timestamp; /* last archival success time */
+ PgStat_Counter failed_count; /* failed archival attempts */
+ char last_failed_wal[MAX_XFN_CHARS + 1]; /* WAL file involved in
+ * last failure */
+ TimestampTz last_failed_timestamp; /* last archival failure time */
+ TimestampTz stat_reset_timestamp;
+} PgStat_Archiver;
/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgDummy
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_alloc;
+ TimestampTz stat_reset_timestamp;
+} PgStat_BgWriter;
/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
+ * PgStat_CheckPointer checkpointer statistics
* ----------
*/
-
-typedef struct PgStat_MsgInquiry
+typedef struct PgStat_CheckPointer
{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgResetslrucounter Sent by the backend to tell the collector
- * to reset a SLRU counter
- * ----------
- */
-typedef struct PgStat_MsgResetslrucounter
-{
- PgStat_MsgHdr m_hdr;
- int m_index;
-} PgStat_MsgResetslrucounter;
-
-/* ----------
- * PgStat_MsgResetreplslotcounter Sent by the backend to tell the collector
- * to reset replication slot counter(s)
- * ----------
- */
-typedef struct PgStat_MsgResetreplslotcounter
-{
- PgStat_MsgHdr m_hdr;
- char m_slotname[NAMEDATALEN];
- bool clearall;
-} PgStat_MsgResetreplslotcounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgWal Sent by backends and background processes to update WAL statistics.
- * ----------
- */
-typedef struct PgStat_MsgWal
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_wal_records;
- PgStat_Counter m_wal_fpi;
- uint64 m_wal_bytes;
- PgStat_Counter m_wal_buffers_full;
-} PgStat_MsgWal;
-
-/* ----------
- * PgStat_MsgSLRU Sent by a backend to update SLRU statistics.
- * ----------
- */
-typedef struct PgStat_MsgSLRU
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_index;
- PgStat_Counter m_blocks_zeroed;
- PgStat_Counter m_blocks_hit;
- PgStat_Counter m_blocks_read;
- PgStat_Counter m_blocks_written;
- PgStat_Counter m_blocks_exists;
- PgStat_Counter m_flush;
- PgStat_Counter m_truncate;
-} PgStat_MsgSLRU;
-
-/* ----------
- * PgStat_MsgReplSlot Sent by a backend or a wal sender to update replication
- * slot statistics.
- * ----------
- */
-typedef struct PgStat_MsgReplSlot
-{
- PgStat_MsgHdr m_hdr;
- char m_slotname[NAMEDATALEN];
- bool m_drop;
- PgStat_Counter m_spill_txns;
- PgStat_Counter m_spill_count;
- PgStat_Counter m_spill_bytes;
- PgStat_Counter m_stream_txns;
- PgStat_Counter m_stream_count;
- PgStat_Counter m_stream_bytes;
-} PgStat_MsgReplSlot;
-
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_CheckPointer;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -548,7 +225,6 @@ typedef struct PgStat_FunctionCounts
*/
typedef struct PgStat_BackendFunctionEntry
{
- Oid f_id;
PgStat_FunctionCounts f_counts;
} PgStat_BackendFunctionEntry;
@@ -564,101 +240,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgResetslrucounter msg_resetslrucounter;
- PgStat_MsgResetreplslotcounter msg_resetreplslotcounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgWal msg_wal;
- PgStat_MsgSLRU msg_slru;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
- PgStat_MsgReplSlot msg_replslot;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -667,13 +250,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9F
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -683,7 +262,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -693,29 +271,86 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatEntryHead common header struct for PgStat_Stat*Entry
+ * ----------
+ */
+typedef struct PgStat_StatEntryHeader
+{
+ LWLock lock;
+ bool dropped; /* This entry is being dropped and should
+ * be removed when refcount goes to
+ * zero. */
+ pg_atomic_uint32 refcount; /* How many backends are referenceing */
+} PgStat_StatEntryHeader;
+
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/* ----------
+ * PgStat_Wal Sent by backends and background processes to
+ * update WAL statistics.
+ * ----------
+ */
+typedef struct PgStat_Wal
+{
+ WalUsage wal_usage;
+ PgStat_Counter wal_buffers_full;
+ TimestampTz stat_reset_timestamp;
+} PgStat_Wal;
+
+/*
+ * SLRU statistics kept in the stats collector
+ */
+typedef struct PgStat_SLRUStats
+{
+ PgStat_Counter blocks_zeroed;
+ PgStat_Counter blocks_hit;
+ PgStat_Counter blocks_read;
+ PgStat_Counter blocks_written;
+ PgStat_Counter blocks_exists;
+ PgStat_Counter flush;
+ PgStat_Counter truncate;
+ TimestampTz stat_reset_timestamp;
+} PgStat_SLRUStats;
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
- Oid tableid;
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ /* Persistent data follow */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -735,99 +370,35 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
{
- Oid functionid;
-
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ /* Persistent data follow */
PgStat_Counter f_numcalls;
PgStat_Counter f_total_time; /* times in microseconds */
PgStat_Counter f_self_time;
} PgStat_StatFuncEntry;
-
-/*
- * Archiver statistics kept in the stats collector
- */
-typedef struct PgStat_ArchiverStats
-{
- PgStat_Counter archived_count; /* archival successes */
- char last_archived_wal[MAX_XFN_CHARS + 1]; /* last WAL file
- * archived */
- TimestampTz last_archived_timestamp; /* last archival success time */
- PgStat_Counter failed_count; /* failed archival attempts */
- char last_failed_wal[MAX_XFN_CHARS + 1]; /* WAL file involved in
- * last failure */
- TimestampTz last_failed_timestamp; /* last archival failure time */
- TimestampTz stat_reset_timestamp;
-} PgStat_ArchiverStats;
-
-/*
- * Global statistics kept in the stats collector
- */
-typedef struct PgStat_GlobalStats
-{
- TimestampTz stats_timestamp; /* time of stats file update */
- PgStat_Counter timed_checkpoints;
- PgStat_Counter requested_checkpoints;
- PgStat_Counter checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter checkpoint_sync_time;
- PgStat_Counter buf_written_checkpoints;
- PgStat_Counter buf_written_clean;
- PgStat_Counter maxwritten_clean;
- PgStat_Counter buf_written_backend;
- PgStat_Counter buf_fsync_backend;
- PgStat_Counter buf_alloc;
- TimestampTz stat_reset_timestamp;
-} PgStat_GlobalStats;
-
-/*
- * WAL statistics kept in the stats collector
- */
-typedef struct PgStat_WalStats
-{
- PgStat_Counter wal_records;
- PgStat_Counter wal_fpi;
- uint64 wal_bytes;
- PgStat_Counter wal_buffers_full;
- TimestampTz stat_reset_timestamp;
-} PgStat_WalStats;
-
-/*
- * SLRU statistics kept in the stats collector
- */
-typedef struct PgStat_SLRUStats
-{
- PgStat_Counter blocks_zeroed;
- PgStat_Counter blocks_hit;
- PgStat_Counter blocks_read;
- PgStat_Counter blocks_written;
- PgStat_Counter blocks_exists;
- PgStat_Counter flush;
- PgStat_Counter truncate;
- TimestampTz stat_reset_timestamp;
-} PgStat_SLRUStats;
-
/*
* Replication slot statistics kept in the stats collector
*/
-typedef struct PgStat_ReplSlotStats
+typedef struct PgStat_ReplSlot
{
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
char slotname[NAMEDATALEN];
PgStat_Counter spill_txns;
PgStat_Counter spill_count;
@@ -836,7 +407,7 @@ typedef struct PgStat_ReplSlotStats
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
TimestampTz stat_reset_timestamp;
-} PgStat_ReplSlotStats;
+} PgStat_ReplSlot;
/* ----------
* Backend states
@@ -885,7 +456,7 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
+ WAIT_EVENT_READING_STATS_FILE,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1138,7 +709,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1335,18 +906,26 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
+
+/*
+ * CheckPointer statistics counters are updated directly by checkpointer and
+ * bufmgr
+ */
+extern PgStat_CheckPointer CheckPointerStats;
/*
* WAL statistics counter is updated by backends and background processes
*/
-extern PgStat_MsgWal WalStats;
+extern PgStat_Wal WalStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1361,33 +940,27 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
extern void pgstat_reset_replslot_counter(const char *name);
+extern void pgstat_report_wal(void);
extern void pgstat_report_autovac(Oid dboid);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples);
@@ -1427,6 +1000,7 @@ extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
extern void pgstat_initstats(Relation rel);
+extern void pgstat_delinkstats(Relation rel);
extern char *pgstat_clip_activity(const char *raw_activity);
@@ -1549,9 +1123,10 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
-extern void pgstat_send_wal(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
+extern void pgstat_report_checkpointer(void);
+extern void pgstat_report_wal(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1560,15 +1135,20 @@ extern void pgstat_send_wal(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_extended(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
-extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
-extern PgStat_GlobalStats *pgstat_fetch_global(void);
-extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
+extern TimestampTz pgstat_get_stat_timestamp(void);
+extern PgStat_Archiver *pgstat_fetch_stat_archiver(void);
+extern PgStat_BgWriter *pgstat_fetch_stat_bgwriter(void);
+extern PgStat_CheckPointer *pgstat_fetch_stat_checkpointer(void);
+extern PgStat_Wal *pgstat_fetch_stat_wal(void);
extern PgStat_SLRUStats *pgstat_fetch_slru(void);
-extern PgStat_ReplSlotStats *pgstat_fetch_replslot(int *nslots_p);
+extern PgStat_ReplSlot *pgstat_fetch_replslot(int *nslots_p);
extern void pgstat_count_slru_page_zeroed(int slru_idx);
extern void pgstat_count_slru_page_hit(int slru_idx);
@@ -1579,5 +1159,6 @@ extern void pgstat_count_slru_flush(int slru_idx);
extern void pgstat_count_slru_truncate(int slru_idx);
extern const char *pgstat_slru_name(int slru_idx);
extern int pgstat_slru_index(const char *name);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index cbf2510fbf..9ed6b54428 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -218,6 +218,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_SHARED_TIDBITMAP,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_PER_XACT_PREDICATE_LIST,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index b9b5c1adda..add9c53ee3 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -88,7 +88,7 @@ enum config_group
PROCESS_TITLE,
STATS,
STATS_MONITORING,
- STATS_COLLECTOR,
+ STATS_ACTIVITY,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index ecb2a366a5..f090f7372a 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -32,6 +32,7 @@ typedef enum TimeoutId
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IDLE_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.27.0
* [PATCH v35 5/7] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 500ms and such large data is
serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 500ms. Locks
on the shared statistics is acquired by the units of such like tables,
functions so the expected chance of collision are not so
high. Furthermore, until 1 second has elapsed since the last flushing
to shared stats, lock failure postpones stats flushing so that lock
contention doesn't slow down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/heap/heapam_handler.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/access/transam/xlog.c | 4 +-
src/backend/catalog/index.c | 24 +-
src/backend/commands/analyze.c | 8 +-
src/backend/commands/dbcommands.c | 2 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/vacuum.c | 4 +-
src/backend/postmaster/autovacuum.c | 74 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 24 +-
src/backend/postmaster/interrupt.c | 5 +-
src/backend/postmaster/pgarch.c | 12 +-
src/backend/postmaster/pgstat.c | 5125 +++++++----------
src/backend/postmaster/postmaster.c | 88 +-
src/backend/replication/basebackup.c | 4 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 4 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/smgr.c | 4 +-
src/backend/tcop/postgres.c | 37 +-
src/backend/utils/adt/pgstatfuncs.c | 59 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 3 -
src/backend/utils/init/postinit.c | 11 +
src/backend/utils/misc/guc.c | 14 +-
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/include/miscadmin.h | 3 +-
src/include/pgstat.h | 581 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/guc_tables.h | 2 +-
src/include/utils/timeout.h | 1 +
34 files changed, 2256 insertions(+), 3872 deletions(-)
inline patch 1: src/backend/access/heap/heapam_handler.c, src/backend/access/heap/vacuumlazy.c, src/backend/access/transam/xlog.c and 31 more (+2256/-3872 lines) download inline-1.patch
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 56b35622f1..80a3c50994 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1059,8 +1059,8 @@ heapam_scan_analyze_next_tuple(TableScanDesc scan, TransactionId OldestXmin,
* our own. In this case we should count and sample the row,
* to accommodate users who load a table and analyze it in one
* transaction. (pgstat_report_analyze has to adjust the
- * numbers we send to the stats collector to make this come
- * out right.)
+ * numbers we report to the activity stats facility to make
+ * this come out right.)
*/
if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(targtuple->t_data)))
{
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3bef0e124b..4ac4e7c03d 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -595,7 +595,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
new_min_multi,
false);
- /* report results to the stats collector, too */
+ /* report results to the activity stats facility, too */
pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
new_live_tuples,
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ca09d81b08..474db9fde3 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8575,9 +8575,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..d33014f915 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1702,28 +1702,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be flushed by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Close relations */
table_close(pg_class, RowExclusiveLock);
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 924ef37c81..06b03cb8e1 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -655,10 +655,10 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
}
/*
- * Report ANALYZE to the stats collector, too. However, if doing
- * inherited stats we shouldn't report, because the stats collector only
- * tracks per-table stats. Reset the changes_since_analyze counter only
- * if we analyzed all columns; otherwise, there is still work for
+ * Report ANALYZE to the activity stats facility, too. However, if doing
+ * inherited stats we shouldn't report, because the activity stats facility
+ * only tracks per-table stats. Reset the changes_since_analyze counter
+ * only if we analyzed all columns; otherwise, there is still work for
* auto-analyze to do.
*/
if (!inh)
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index f27c3fe8c1..de0c749570 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -971,7 +971,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
DropDatabaseBuffers(db_id);
/*
- * Tell the stats collector to forget it immediately, too.
+ * Tell the active stats facility to forget it immediately, too.
*/
pgstat_drop_database(db_id);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index f80a9e96a9..e7ccc6eba7 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -336,10 +336,10 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
/*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
+ * Inform activity stats facility about our activity: basically, we
+ * truncated the matview and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
*/
pgstat_count_truncate(matviewRel);
if (!stmt->skipData)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..3953964038 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -319,8 +319,8 @@ vacuum(List *relations, VacuumParams *params,
errmsg("VACUUM option DISABLE_PAGE_SKIPPING cannot be used with FULL")));
/*
- * Send info about dead objects to the statistics collector, unless we are
- * in autovacuum --- autovacuum.c does this for itself.
+ * Send info about dead objects to the activity statistics facility, unless
+ * we are in autovacuum --- autovacuum.c does this for itself.
*/
if ((params->options & VACOPT_VACUUM) && !IsAutoVacuumWorkerProcess())
pgstat_vacuum_stat();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 9c7d4b0c60..374ddb5d60 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -338,9 +338,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1664,12 +1661,12 @@ AutoVacWorkerMain(int argc, char *argv[])
char dbname[NAMEDATALEN];
/*
- * Report autovac startup to the stats collector. We deliberately do
- * this before InitPostgres, so that the last_autovac_time will get
- * updated even if the connection attempt fails. This is to prevent
- * autovac from getting "stuck" repeatedly selecting an unopenable
- * database, rather than making any progress on stuff it can connect
- * to.
+ * Report autovac startup to the activity stats facility. We
+ * deliberately do this before InitPostgres, so that the
+ * last_autovac_time will get updated even if the connection attempt
+ * fails. This is to prevent autovac from getting "stuck" repeatedly
+ * selecting an unopenable database, rather than making any progress on
+ * stuff it can connect to.
*/
pgstat_report_autovac(dbid);
@@ -1937,8 +1934,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1957,17 +1952,11 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
/*
- * Clean up any dead statistics collector entries for this DB. We always
+ * Clean up any dead activity statistics entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
* nothing worth vacuuming in the database.
*/
@@ -2010,9 +1999,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2091,8 +2077,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2175,8 +2161,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2735,29 +2721,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2778,17 +2741,12 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
bool doanalyze;
autovac_table *tab = NULL;
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
bool wraparound;
AutoVacOpts *avopts;
/* use fresh stats */
autovac_refresh_stats();
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* fetch the relation's relcache entry */
classTup = SearchSysCacheCopy1(RELOID, ObjectIdGetDatum(relid));
if (!HeapTupleIsValid(classTup))
@@ -2812,8 +2770,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
}
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2935,7 +2893,7 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
*
* For analyze, the analysis done is that the number of tuples inserted,
* deleted and updated since the last analyze exceeds a threshold calculated
- * in the same fashion as above. Note that the collector actually stores
+ * in the same fashion as above. Note that the activity statistics stores
* the number of tuples (both live and dead) that there were as of the last
* analyze. This is asymmetric to the VACUUM case.
*
@@ -2945,8 +2903,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
*
* A table whose autovacuum_enabled option is false is
* automatically skipped (unless we have to vacuum it due to freeze_max_age).
- * Thus autovacuum can be disabled for specific tables. Also, when the stats
- * collector does not have data about a table, it will be skipped.
+ * Thus autovacuum can be disabled for specific tables. Also, when the activity
+ * statistics does not have data about a table, it will be skipped.
*
* A table whose vac_base_thresh value is < 0 takes the base value from the
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..4382b1726f 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -234,9 +234,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Send off activity statistics to the activity stats facility
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..dd36b4965e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -350,7 +350,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ BgWriterStats.requested_checkpoints++;
}
/*
@@ -364,7 +364,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ BgWriterStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -486,13 +486,13 @@ CheckpointerMain(void)
CheckArchiveTimeout();
/*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
+ * Send off activity statistics to the activity stats facility. (The
+ * reason why we re-use bgwriter-related code for this is that the
+ * bgwriter and checkpointer used to be just one process. It's
+ * probably not worth the trouble to split the stats support into two
+ * independent stats message types.)
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* If any checkpoint flags have been set, redo the loop to handle the
@@ -698,9 +698,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Report interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1245,8 +1245,8 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ BgWriterStats.buf_written_backend += CheckpointerShmem->num_backend_writes;
+ BgWriterStats.buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index 3d02439b79..53844eb8bb 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -93,9 +93,8 @@ SignalHandlerForCrashExit(SIGNAL_ARGS)
* shut down and exit.
*
* Typically, this handler would be used for SIGTERM, but some procesess use
- * other signals. In particular, the checkpointer exits on SIGUSR2, the
- * stats collector on SIGQUIT, and the WAL writer exits on either SIGINT
- * or SIGTERM.
+ * other signals. In particular, the checkpointer exits on SIGUSR2, and the WAL
+ * writer exits on either SIGINT or SIGTERM.
*
* ShutdownRequestPending should be checked at a convenient place within the
* main loop, or else the main loop should call HandleMainLoopInterrupts.
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 063d1323ea..08fe87341c 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -372,20 +372,20 @@ pgarch_ArchiverCopyLoop(void)
pgarch_archiveDone(xlog);
/*
- * Tell the collector about the WAL file that we successfully
- * archived
+ * Tell the activity statistics facility about the WAL file
+ * that we successfully archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
else
{
/*
- * Tell the collector about the WAL file that we failed to
- * archive
+ * Tell the activity statistics facility about the WAL file
+ * that we failed to archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 309378ae54..76b59df408 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (1000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (250ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (10000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,68 +35,43 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 1000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 250 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 10000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -116,7 +86,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -131,15 +100,24 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
-/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ dsa_pointer global_stats; /* DSA pointer to global stats */
+ dsa_pointer archiver_stats; /* Ditto for archiver stats */
+ dsa_pointer slru_stats; /* Ditto for SLRU stats */
+ int refcount; /* # of processes that is attaching the shared
+ * stats memory */
+} StatsShmemStruct;
+
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
/*
* List of SLRU names that we keep stats for. There is no central registry of
@@ -160,72 +138,159 @@ static const char *const slru_names[] = {
#define SLRU_NUM_ELEMENTS lengthof(slru_names)
/*
- * SLRU statistics counts waiting to be sent to the collector. These are
- * stored directly in stats message format so they can be sent without needing
- * to copy things around. We assume this variable inits to zeroes. Entries
- * are one-to-one with slru_names[].
+ * SLRU statistics counts waiting to be written to the shared activity
+ * statistics. We assume this variable inits to zeroes. Entries are
+ * one-to-one with slru_names[].
+ * Changes of SLRU counters are reported within critical sections so we use
+ * static memory in order to avoid memory allocation.
*/
-static PgStat_MsgSLRU SLRUStats[SLRU_NUM_ELEMENTS];
+static PgStat_StatSLRUEntry local_SLRUStats[SLRU_NUM_ELEMENTS];
+static bool have_slrustats = false;
/* ----------
* Local data
* ----------
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
-
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
+ * Enums and types to define shared statistics structure.
+ *
+ * Statistics entries for each object is stored in individual DSA-allocated
+ * memory. Every entry is pointed from the dshash pgStatSharedHash via
+ * dsa_pointer. The structure makes object-stats entries not moved by dshash
+ * resizing, and allows the dshash can release lock sooner on stats
+ * updates. Also it reduces interfering among write-locks on each stat entry by
+ * not relying on partition lock of dshash. PgStatLocalHashEntry is the
+ * local-stats equivalent of PgStatHashEntry for shared stat entries.
+ *
+ * Each stat entry is enveloped by the type PgStatEnvelope, which stores common
+ * attribute of all kind of statistics and a LWLock lock object.
+ *
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStatEnvelope (dsa memory block)
*
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Local stats are stored as:
+ *
+ * dshash pgStatLocalHash
+ * -> PgStatLocalHashEntry (dynahash entry)
+ * (direct pointer)-> PgStatEnvelope (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries. */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_ALL, /* Not a type, for the parameters of
+ * pgstat_collect_stat_entries */
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION /* per-function statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static size_t pgstat_entsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry) /* PGSTAT_TYPE_FUNCTION */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static size_t pgstat_localentsize[] =
+{
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry) /* PGSTAT_TYPE_FUNCTION */
+};
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashEntryKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object OID */
+} PgStatHashEntryKey;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Stats numbers that are waiting for flushing out to shared stats are held in
+ * pgStatLocalHash,
*/
-static HTAB *pgStatFunctions = NULL;
+typedef struct PgStatHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ dsa_pointer env; /* pointer to shared stats envelope in DSA */
+} PgStatHashEntry;
+
+/* struct for shared statistics entry pointed from shared hash entry. */
+typedef struct PgStatEnvelope
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* databaseid */
+ Oid objectid; /* objectid */
+ size_t len; /* length of body, fixed per type. */
+ LWLock lock; /* lightweight lock to protect body */
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatEnvelope;
+
+#define PgStatEnvelopeSize(bodylen) \
+ (offsetof(PgStatEnvelope, body) + (bodylen))
+
+/* struct for shared SLRU stats */
+typedef struct PgStatSharedSLRUStats
+{
+ PgStat_StatSLRUEntry entry[SLRU_NUM_ELEMENTS];
+ LWLock lock;
+} PgStatSharedSLRUStats;
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ PgStatEnvelope *env; /* pointer to stats envelope in heap */
+} PgStatLocalHashEntry;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * Snapshot is stats entry that is locally copied to offset stable values for a
+ * transaction.
*/
-static bool have_function_stats = false;
+typedef struct PgStatSnapshot
+{
+ PgStatHashEntryKey key;
+ bool negative;
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatSnapshot;
+
+#define PgStatSnapshotSize(bodylen) \
+ (offsetof(PgStatSnapshot, body) + (bodylen))
+
+/* parameter for shared hashes */
+static const dshash_parameters dsh_rootparams = {
+ sizeof(PgStatHashEntryKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
+
+/* Local stats numbers are stored here */
+static HTAB *pgStatLocalHash = NULL;
+
+#define HAVE_ANY_PENDING_STATS() \
+ (pgStatLocalHash != NULL || \
+ pgStatXactCommit != 0 || pgStatXactRollback != 0)
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -262,11 +327,10 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatSnapshotHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -275,20 +339,19 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are collected not per database nor per table
+ * basis. shared_* points to shared memory and snapshot_* are backend
+ * snapshots.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+static bool global_snapshot_is_valid = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats snapshot_globalStats;
+static PgStatSharedSLRUStats *shared_SLRUStats;
+static PgStat_StatSLRUEntry snapshot_SLRUStats[SLRU_NUM_ELEMENTS];
/*
* Total time charged to functions so far in the current backend.
@@ -297,526 +360,278 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/*
+ * Newly created shared stats entries needs to be initialized before the other
+ * processes get access it. get_stat_entry() calls it for the purpose.
+ */
+typedef void (*entry_initializer) (PgStatEnvelope * env);
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStatEnvelope * get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait,
+ entry_initializer initfunc, bool *found);
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid);
+static void create_missing_dbentries(void);
+static void pgstat_write_database_stats(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry);
+
+static bool flush_tabstat(PgStatEnvelope * env, bool nowait);
+static bool flush_funcstat(PgStatEnvelope * env, bool nowait);
+static bool flush_dbstat(PgStatEnvelope * env, bool nowait);
+static bool flush_slrustat(bool nowait);
+
+static void init_tabentry(PgStatEnvelope * env);
+static void init_funcentry(PgStatEnvelope * env);
+static void init_dbentry(PgStatEnvelope * env);
+
+static bool delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait);
+static PgStatEnvelope * get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static void pgstat_snapshot_global_stats(void);
+
static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static void pgstat_send_slru(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
-
-static void pgstat_setup_memcxt(void);
-
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_slru(PgStat_MsgSLRU *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
+ bool found;
-#define TESTBYTEVAL ((char) 199)
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
- {
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
- }
-
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
+ if (!IsUnderPostmaster)
{
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
-
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
-
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+ Assert(!found);
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ }
+}
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgStatLocalContext and pgStatSnapshotContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgStatLocalContext)
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+
+ if (!pgStatSnapshotContext)
+ pgStatSnapshotContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Database statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+}
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read saved statistics files if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
- test_byte++; /* just make sure variable is changed */
+ /*
+ * Don't use dsm under postmaster, or when not tracking counts.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ pgstat_setup_memcxt();
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ if (area)
+ return;
- /* If we get here, we have a working socket */
- break;
- }
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that processes going to exit can find whether it
+ * is the last or not.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
+
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_rootparams, 0);
+
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->slru_stats =
+ dsa_allocate0(area, sizeof(PgStatSharedSLRUStats));
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
+
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
+
+ shared_SLRUStats = (PgStatSharedSLRUStats *)
+ dsa_get_address(area, StatsShmem->slru_stats);
+ LWLockInitialize(&shared_SLRUStats->lock, LWTRANCHE_STATS);
+
+ /* Load saved data if any. */
+ pgstat_read_statsfiles();
+
+ StatsShmem->refcount = 1;
}
+ LWLockRelease(StatsLock);
+
/*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
+ * If we're not the first process, attach existing shared stats area
+ * outside the StatsLock section.
*/
+ if (!area)
{
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
+ /* Attach shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_rootparams,
+ StatsShmem->hash_handle, 0);
+
+ /* Setup local variables */
+ pgStatLocalHash = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
+ shared_SLRUStats = (PgStatSharedSLRUStats *)
+ dsa_get_address(area, StatsShmem->slru_stats);
}
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+ global_snapshot_is_valid = false;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_stats)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
- {
- int nchars;
- Oid tmp_oid;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount < 1)
+ {
/*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
+ * The process is the last one that is attaching the shared stats
+ * memory. Write out the stats files if requested.
*/
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
+ if (write_stats)
+ pgstat_write_statsfiles();
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ /* No one is using the area. */
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
- FreeDir(dir);
+
+ LWLockRelease(StatsLock);
+
+ /*
+ * Detach the area. Automatically destroyed when the last process detached
+ * it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatLocalHash = NULL;
+ global_snapshot_is_valid = false;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
-#ifdef EXEC_BACKEND
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
-
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
/*
- * Okay, fork off the collector.
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
+ detach_shared_stats(false); /* Don't write files. */
+ attach_shared_stats();
}
/* ------------------------------------------------------------
@@ -824,147 +639,491 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait = !force; /* Don't use force ever after */
+ HASH_SEQ_STATUS scan;
+ PgStatLocalHashEntry *lent;
+ PgStatLocalHashEntry **dbentlist;
+ int dbentlistlen = 8;
+ int ndbentries = 0;
+ int remains = 0;
int i;
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (area == NULL || !HAVE_ANY_PENDING_STATS())
+ return 0;
+
+ dbentlist = palloc(sizeof(PgStatLocalHashEntry *) * dbentlistlen);
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
+ if (nowait)
+ {
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
+ {
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ nowait = false;
+ }
/*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * flush_tabstat applies some of stats numbers of flushed entries into
+ * local database stats. So flush-out database stats later.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ if (pgStatLocalHash)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /* Step 1: flush out other than database stats */
+ hash_seq_init(&scan, pgStatLocalHash);
+ while ((lent = (PgStatLocalHashEntry *) hash_seq_search(&scan)) != NULL)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
+ bool remove = false;
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
+ switch (lent->env->type)
+ {
+ case PGSTAT_TYPE_DB:
+ if (ndbentries >= dbentlistlen)
+ {
+ dbentlistlen *= 2;
+ dbentlist = repalloc(dbentlist,
+ sizeof(PgStatLocalHashEntry *) *
+ dbentlistlen);
+ }
+ dbentlist[ndbentries++] = lent;
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent->env, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent->env, nowait))
+ remove = true;
+ break;
+ default:
+ Assert(false);
+ }
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
+ }
+
+ /* Step 2: flush out database stats */
+ for (i = 0; i < ndbentries; i++)
+ {
+ PgStatLocalHashEntry *lent = dbentlist[i];
+
+ if (flush_dbstat(lent->env, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ pfree(dbentlist);
+
+ if (remains <= 0)
+ {
+ hash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
}
+ /* flush SLRU stats */
+ flush_slrustat(nowait);
+
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (shared_globalStats->stats_timestamp < now)
+ shared_globalStats->stats_timestamp = now;
+ LWLockRelease(StatsLock);
+
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * If we have pending local stats, let the caller know the retry interval.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
+ if (HAVE_ANY_PENDING_STATS())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
+
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
- /* Finally send SLRU statistics */
- pgstat_send_slru();
+ return retry_interval / 1000;
+ }
+
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
}
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_tabstat(PgStatEnvelope * lenv, bool nowait)
+{
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+ bool found;
+
+ Assert(lenv->type == PGSTAT_TYPE_TABLE);
+
+ lstats = (PgStat_TableStatus *) &lenv->body;
+ dboid = lstats->t_shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_TABLE, dboid, lstats->t_id,
+ nowait, init_tabentry, &found);
+
+ /* skip if dshash failed to acquire lock */
+ if (shenv == NULL)
+ return false;
+
+ /* retrieve the shared table stats entry from the envelope */
+ shtabstats = (PgStat_StatTabEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated ||
+ lstats->t_counts.vacuum_count > 0 ||
+ lstats->t_counts.analyze_count > 0 ||
+ lstats->t_counts.autovac_vacuum_count > 0 ||
+ lstats->t_counts.autovac_analyze_count > 0)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ /* clear the change counter if requested */
+ if (lstats->t_counts.reset_changed_tuples)
+ shtabstats->changes_since_analyze = 0;
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /*
+ * Update vacuum/analyze timestamp and counters, so that the values won't
+ * goes back.
+ */
+ if (shtabstats->vacuum_timestamp < lstats->vacuum_timestamp)
+ shtabstats->vacuum_timestamp = lstats->vacuum_timestamp;
+ shtabstats->vacuum_count += lstats->t_counts.vacuum_count;
+
+ if (shtabstats->autovac_vacuum_timestamp < lstats->autovac_vacuum_timestamp)
+ shtabstats->autovac_vacuum_timestamp = lstats->autovac_vacuum_timestamp;
+ shtabstats->autovac_vacuum_count += lstats->t_counts.autovac_vacuum_count;
+
+ if (shtabstats->analyze_timestamp < lstats->analyze_timestamp)
+ shtabstats->analyze_timestamp = lstats->analyze_timestamp;
+ shtabstats->analyze_count += lstats->t_counts.analyze_count;
+
+ if (shtabstats->autovac_analyze_timestamp < lstats->autovac_analyze_timestamp)
+ shtabstats->autovac_analyze_timestamp = lstats->autovac_analyze_timestamp;
+ shtabstats->autovac_analyze_count += lstats->t_counts.autovac_analyze_count;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shenv->lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ return true;
+}
+
+/* ----------
+ * init_tabentry() -
+ *
+ * initializes table stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
*/
static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+init_tabentry(PgStatEnvelope * env)
{
- int n;
- int len;
+ PgStat_StatTabEntry *tabent = (PgStat_StatTabEntry *) &env->body;
+
+ /*
+ * If it's a new table entry, initialize counters to the values we just
+ * got.
+ */
+ Assert(env->type == PGSTAT_TYPE_TABLE);
+ tabent->tableid = env->objectid;
+ tabent->numscans = 0;
+ tabent->tuples_returned = 0;
+ tabent->tuples_fetched = 0;
+ tabent->tuples_inserted = 0;
+ tabent->tuples_updated = 0;
+ tabent->tuples_deleted = 0;
+ tabent->tuples_hot_updated = 0;
+ tabent->n_live_tuples = 0;
+ tabent->n_dead_tuples = 0;
+ tabent->changes_since_analyze = 0;
+ tabent->blocks_fetched = 0;
+ tabent->blocks_hit = 0;
+
+ tabent->vacuum_timestamp = 0;
+ tabent->vacuum_count = 0;
+ tabent->autovac_vacuum_timestamp = 0;
+ tabent->autovac_vacuum_count = 0;
+ tabent->analyze_timestamp = 0;
+ tabent->analyze_count = 0;
+ tabent->autovac_analyze_timestamp = 0;
+ tabent->autovac_analyze_count = 0;
+}
+
+
+/*
+ * flush_funcstat - flush out a local function stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatEnvelope * env, bool nowait)
+{
+ /* we assume this inits to all zeroes: */
+ static const PgStat_FunctionCounts all_zeroes;
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatFuncEntry *sharedent = NULL; /* shared stats entry */
+ bool found;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) &env->body;
+
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&localent->f_counts, &all_zeroes,
+ sizeof(PgStat_FunctionCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, localent->f_id,
+ nowait, init_funcentry, &found);
+ /* skip if dshash failed to acquire lock */
+ if (shenv == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* retrieve the shared table stats entry from the envelope */
+ sharedent = (PgStat_StatFuncEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->f_numcalls += localent->f_counts.f_numcalls;
+ sharedent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ sharedent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shenv->lock);
+
+ return true;
+}
+
+
+/* ----------
+ * init_funcentry() -
+ *
+ * initializes function stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_funcentry(PgStatEnvelope * env)
+{
+ PgStat_StatFuncEntry *shstat = (PgStat_StatFuncEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ shstat->functionid = env->objectid;
+ shstat->f_numcalls = 0;
+ shstat->f_total_time = 0;
+ shstat->f_self_time = 0;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatEnvelope * env, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStatEnvelope *shenv;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &env->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_DB, localent->databaseid, InvalidOid,
+ nowait, init_dbentry, NULL);
+
+ /* skip if dshash failed to acquire lock */
+ if (!shenv)
+ return false;
+
+ /* retrieve the shared stats entry from the envelope */
+ sharedent = (PgStat_StatDBEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(localent->databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -972,257 +1131,149 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&shenv->lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
-/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+
+/* ----------
+ * init_dbentry() -
+ *
+ * initializes database stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
*/
static void
-pgstat_send_funcstats(void)
+init_dbentry(PgStatEnvelope * env)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+ dbentry->databaseid = env->databaseid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ /* initialize the new shared entry */
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
+}
+
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
+/*
+ * flush_slrustat - flush out a local SLRU stats entries
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if all local SLRU stats are successfully flushed out.
+ */
+static bool
+flush_slrustat(bool nowait)
+{
+ int i;
- if (pgStatFunctions == NULL)
- return;
+ if (!have_slrustats)
+ return true;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shared_SLRUStats->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shared_SLRUStats->lock, LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS ; i++)
{
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
+ PgStat_StatSLRUEntry *sharedent = &shared_SLRUStats->entry[i];
+ PgStat_StatSLRUEntry *localent = &local_SLRUStats[i];
+
+ sharedent->blocks_zeroed += localent->blocks_zeroed;
+ sharedent->blocks_hit += localent->blocks_hit;
+ sharedent->blocks_read += localent->blocks_read;
+ sharedent->blocks_written += localent->blocks_written;
+ sharedent->blocks_exists += localent->blocks_exists;
+ sharedent->flush += localent->flush;
+ sharedent->truncate += localent->truncate;
+ }
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
+ /* done, clear the local entry */
+ MemSet(local_SLRUStats, 0,
+ sizeof(PgStat_StatSLRUEntry) * SLRU_NUM_ELEMENTS);
+ LWLockRelease(&shared_SLRUStats->lock);
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
+ have_slrustats = false;
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
+ return true;
+}
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- have_function_stats = false;
+/*
+ * Create the filename for a DB stat file; filename is output parameter points
+ * to a character buffer of length len.
+ */
+static void
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
+{
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
}
/* ----------
- * pgstat_vacuum_stat() -
+ * collect_stat_entries() -
*
- * Will tell the collector about objects he can get rid of.
+ * Collect the shared statistics entries specified by type and dbid. Returns a
+ * list of pointer to shared statistics in palloc'ed memory. If type is
+ * PGSTAT_TYPE_ALL, all types of statistics of the database is collected. If
+ * type is PGSTAT_TYPE_DB, the parameter dbid is ignored and collect all
+ * PGSTAT_TYPE_DB entries.
* ----------
*/
-void
-pgstat_vacuum_stat(void)
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
- return;
-
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
-
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ int listlen = 16;
+ PgStatEnvelope **envlist = palloc(sizeof(PgStatEnvelope * *) * listlen);
+ int n = 0;
+
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ if ((type != PGSTAT_TYPE_ALL && p->key.type != type) ||
+ (type != PGSTAT_TYPE_DB && p->key.databaseid != dbid))
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
+ if (n >= listlen - 1)
{
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
+ listlen *= 2;
+ envlist = repalloc(envlist, listlen * sizeof(PgStatEnvelope * *));
}
+ envlist[n++] = dsa_get_address(area, p->env);
}
+ dshash_seq_term(&hstat);
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
+ envlist[n] = NULL;
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
- }
+ return envlist;
}
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
* into a temporary hash table. Caller should hash_destroy the result
@@ -1231,7 +1282,7 @@ pgstat_vacuum_stat(void)
* ----------
*/
static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
HTAB *htab;
HASHCTL hash_ctl;
@@ -1245,7 +1296,7 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
hash_ctl.entrysize = sizeof(Oid);
hash_ctl.hcxt = CurrentMemoryContext;
htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
+ PGSTAT_TABLE_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
@@ -1272,65 +1323,185 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
}
+/* ----------
+ * pgstat_vacuum_stat() -
+ *
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
+ * ----------
+ */
+void
+pgstat_vacuum_stat(void)
+{
+ HTAB *dbids; /* database ids */
+ HTAB *relids; /* relation ids in the current database */
+ HTAB *funcids; /* function ids in the current database */
+ PgStatEnvelope **victims; /* victim entry list */
+ int arraylen = 0; /* storage size of the above */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+ int i;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+
+ /* collect victims from shared stats */
+ arraylen = 16;
+ victims = palloc(sizeof(PgStatEnvelope * *) * arraylen);
+ nvictims = 0;
+
+ dshash_seq_init(&dshstat, pgStatSharedHash, false);
+
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ HTAB *oidtab;
+ Oid *key;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
+ continue;
+
+ switch (ent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidtab = dbids;
+ key = &ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidtab = relids;
+ key = &ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidtab = funcids;
+ key = &ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_ALL:
+ Assert(false);
+ break;
+ }
+
+ /* Skip existent objects. */
+ if (hash_search(oidtab, key, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* extend the list if needed */
+ if (nvictims >= arraylen)
+ {
+ arraylen *= 2;
+ victims = repalloc(victims, sizeof(PgStatEnvelope * *) * arraylen);
+ }
+
+ victims[nvictims++] = dsa_get_address(area, ent->env);
+ }
+ dshash_seq_term(&dshstat);
+ hash_destroy(dbids);
+ hash_destroy(relids);
+ hash_destroy(funcids);
+
+ /* Now try removing the victim entries */
+ for (i = 0; i < nvictims; i++)
+ {
+ PgStatEnvelope *p = victims[i];
+
+ delete_stat_entry(p->type, p->databaseid, p->objectid, true);
+ }
+}
+
+
+/* ----------
+ * delete_stat_entry() -
+ *
+ * Deletes the specified entry from shared stats hash.
+ *
+ * Returns true when successfully deleted.
+ * ----------
+ */
+static bool
+delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait)
+{
+ PgStatHashEntryKey key;
+ PgStatHashEntry *ent;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ ent = dshash_find_extended(pgStatSharedHash, &key,
+ true, nowait, false, NULL);
+
+ if (!ent)
+ return false; /* lock failed or not found */
+
+ /* The entry is exclusively locked, so we can free the chunk first. */
+ dsa_free(area, ent->env);
+ dshash_delete_entry(pgStatSharedHash, ent);
+
+ return true;
+}
+
+
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
+ Assert(OidIsValid(databaseid));
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
+ for (p = envlist; *p != NULL; p++)
+ delete_stat_entry((*p)->type, (*p)->databaseid, (*p)->objectid, true);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ pfree(envlist);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1339,20 +1510,47 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ /* Lookup the entries of the current database in the stats hash. */
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
+ for (p = envlist; *p != NULL; p++)
+ {
+ PgStatEnvelope *env = *p;
+ PgStat_StatDBEntry *dbstat;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+
+ switch (env->type)
+ {
+ case PGSTAT_TYPE_TABLE:
+ init_tabentry(env);
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ init_funcentry(env);
+ break;
+
+ case PGSTAT_TYPE_DB:
+ init_dbentry(env);
+ dbstat = (PgStat_StatDBEntry *) &env->body;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ break;
+ default:
+ Assert(false);
+ }
+
+ LWLockRelease(&env->lock);
+ }
+
+ pfree(envlist);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1361,29 +1559,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ /* Reset the bgwriter statistics for the cluster. */
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1392,17 +1598,42 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ env = get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid,
+ false, NULL, NULL);
+ Assert(env);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&env->lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ env = get_stat_entry(stattype, MyDatabaseId, objoid, false, NULL, NULL);
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ if (env->type == PGSTAT_TYPE_TABLE)
+ init_tabentry(env);
+ else
+ {
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ init_funcentry(env);
+ }
+ LWLockRelease(&env->lock);
}
/* ----------
@@ -1418,15 +1649,27 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_reset_slru_counter(const char *name)
{
- PgStat_MsgResetslrucounter msg;
+ int i;
+ TimestampTz ts = GetCurrentTimestamp();
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSLRUCOUNTER);
- msg.m_index = (name) ? pgstat_slru_index(name) : -1;
-
- pgstat_send(&msg, sizeof(msg));
+ if (name)
+ {
+ i = pgstat_slru_index(name);
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(&shared_SLRUStats[i], 0, sizeof(PgStat_StatSLRUEntry));
+ shared_SLRUStats->entry[i].stat_reset_timestamp = ts;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS; i++)
+ {
+ MemSet(&shared_SLRUStats[i], 0, sizeof(PgStat_StatSLRUEntry));
+ shared_SLRUStats->entry[i].stat_reset_timestamp = ts;
+ }
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
@@ -1440,48 +1683,63 @@ pgstat_reset_slru_counter(const char *name)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ ts = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Store the last autovacuum time in the database's hash table entry.
+ */
+ dbentry = get_local_dbstat_entry(dboid);
+ dbentry->last_autovac_time = ts;
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_TableStatus *tabentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+ tabentry = get_local_tabstat_entry(tableoid, shared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->t_counts.autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->t_counts.vacuum_count++;
+ }
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1492,9 +1750,10 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_TableStatus *tabentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1502,10 +1761,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1523,158 +1782,172 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ tabentry = get_local_tabstat_entry(RelationGetRelid(rel),
+ rel->rd_rel->relisshared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->t_counts.reset_changed_tuples = true;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.analyze_count++;
+ }
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-
-/* --------
- * pgstat_report_checksum_failures_in_db() -
- *
- * Tell the collector about one or more checksum failures.
- * --------
- */
-void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
-{
- PgStat_MsgChecksumFailure msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
-
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
-/* ----------
- * pgstat_ping() -
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Send some junk data to the collector to increase traffic.
- * ----------
+ * Reports about one or more checksum failures.
+ * --------
*/
void
-pgstat_ping(void)
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgDummy msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dbentry = get_local_dbstat_entry(dboid);
+
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
{
+ PgStatEnvelope *env;
PgStat_BackendFunctionEntry *htabent;
bool found;
@@ -1685,26 +1958,15 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
+ htabent = (PgStat_BackendFunctionEntry *) &env->body;
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
if (!found)
MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent->f_id = fcinfo->flinfo->fn_oid;
+
fcu->fs = &htabent->f_counts;
/* save stats for this function, later used to compensate for recursion */
@@ -1717,31 +1979,38 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
+ PgStatEnvelope *env;
+
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+ if (!env)
return NULL;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ return (PgStat_BackendFunctionEntry *) &env->body;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1782,9 +2051,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1796,8 +2062,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1817,7 +2082,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1833,116 +2099,157 @@ pgstat_initstats(Relation rel)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
-/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize body part of the returned envelope.
+ * ----------
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+static PgStatEnvelope *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
+ PgStatHashEntryKey key;
+ PgStatLocalHashEntry *entry;
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ if (pgStatLocalHash == NULL)
{
HASHCTL ctl;
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = sizeof(PgStatLocalHashEntry);
+
+ pgStatLocalHash = hash_create("Local stat entries",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ entry = hash_search(pgStatLocalHash, &key,
+ create ? HASH_ENTER : HASH_FIND, found);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ {
+ int len = pgstat_localentsize[type];
+
+ entry->env = MemoryContextAlloc(TopMemoryContext,
+ PgStatEnvelopeSize(len));
+ entry->env->type = type;
+ entry->env->len = len;
}
+ return entry->env;
+}
+
+/* ----------
+ * get_local_dbstat_entry() -
+ *
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
+ */
+static PgStat_StatDBEntry *
+get_local_dbstat_entry(Oid dbid)
+{
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ bool found;
+
/*
* Find an entry or create a new one.
*/
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
+ env = get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
if (!found)
{
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
+ dbentry->databaseid = dbid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
}
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
+ return dbentry;
+}
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
+ */
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
+{
+ PgStatEnvelope *env;
+ PgStat_TableStatus *tabentry;
+ bool found;
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- return entry;
+ tabentry = (PgStat_TableStatus *) &env->body;
+
+ if (!found)
+ {
+ tabentry->t_id = rel_id;
+ tabentry->t_shared = isshared;
+ tabentry->trans = NULL;
+ MemSet(&tabentry->t_counts, 0, sizeof(PgStat_TableCounts));
+ tabentry->vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ }
+
+ return tabentry;
}
+
/*
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel from the current
+ * database then from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry, return NULL, don't create a new one
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStatEnvelope *env;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!env)
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
+ if (env)
+ return (PgStat_TableStatus *) &env->body;
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return NULL;
}
/*
@@ -2373,8 +2680,8 @@ AtPrepare_PgStat(void)
*
* All we need do here is unlink the transaction stats state from the
* nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
+ * reported to the activity stats facility immediately, while the effects on
+ * live and dead tuple counts are preserved in the 2PC state file.
*
* Note: AtEOXact_PgStat is not called during PREPARE.
*/
@@ -2419,7 +2726,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2455,7 +2762,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2472,88 +2779,176 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
}
+/* ----------
+ * snapshot_statentry() -
+ *
+ * Common routine for functions pgstat_fetch_stat_*entry()
+ *
+ * Returns the pointer to the snapshot of the shared entry for the key or NULL
+ * if not found. Returned snapshots are stable during the current transaction
+ * or until pgstat_clear_snapshot() is called.
+ *
+ * Created snapshots are stored in pgStatSnapshotHash.
+ */
+static void *
+snapshot_statentry(const PgStatTypes type, const Oid dbid, const Oid objid)
+{
+ PgStatSnapshot *snap = NULL;
+ bool found;
+ PgStatHashEntryKey key;
+ size_t statentsize = pgstat_entsize[type];
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ /*
+ * Create new hash, with rather arbitrary initial number of entries since
+ * we don't know how this hash will grow.
+ */
+ if (!pgStatSnapshotHash)
+ {
+ HASHCTL ctl;
+
+ /*
+ * Create the hash in the stats context
+ *
+ * The entry is prepended by common header part represented by
+ * PgStatSnapshot.
+ */
+
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = PgStatSnapshotSize(statentsize);
+ ctl.hcxt = pgStatSnapshotContext;
+ pgStatSnapshotHash = hash_create("pgstat snapshot hash", 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ /* Find a snapshot */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ snap = hash_search(pgStatSnapshotHash, &key, HASH_ENTER, &found);
+
+ /*
+ * Refer shared hash if not found in the snapshot hash.
+ *
+ * In transaction state, it is obvious that we should create a snapshot
+ * entriy for consistency. If we are not, we return an up-to-date entry.
+ * Having said that, we need a snapshot since shared stats entry can be
+ * modified anytime. We share the same snapshot entry for the purpose.
+ */
+ if (!found || !IsTransactionState())
+ {
+ PgStatEnvelope *shenv;
+
+ shenv = get_stat_entry(type, dbid, objid, true, NULL, NULL);
+
+ if (shenv)
+ memcpy(&snap->body, &shenv->body, statentsize);
+
+ snap->negative = !shenv;
+ }
+
+ if (snap->negative)
+ return NULL;
+
+ return &snap->body;
+}
+
+
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *)
+ snapshot_statentry(PGSTAT_TYPE_DB, dbid, InvalidOid);
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_snapshot() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_snapshot(bool shared, Oid reloid)
+{
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(PGSTAT_TYPE_TABLE, dboid, reloid);
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2567,25 +2962,60 @@ pgstat_fetch_stat_tabentry(Oid relid)
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- /* load the stats file if needed */
- backend_read_statsfile();
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, func_id);
+}
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ * ----------
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+ int i;
+
+ attach_shared_stats();
+
+ /* Nothing to do if already done */
+ if (global_snapshot_is_valid)
+ return;
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+
+ memcpy(&snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+ memcpy(&snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ memcpy(snapshot_SLRUStats, shared_SLRUStats,
+ sizeof(PgStat_StatSLRUEntry) * SLRU_NUM_ELEMENTS);
+
+ LWLockRelease(StatsLock);
+
+ /* Fill in empty timestamp of SLRU stats */
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS ; i++)
{
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ if (snapshot_SLRUStats[i].stat_reset_timestamp == 0)
+ snapshot_SLRUStats[i].stat_reset_timestamp =
+ snapshot_globalStats.stat_reset_timestamp;
}
+ global_snapshot_is_valid = true;
- return funcentry;
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
}
-
/* ----------
* pgstat_fetch_stat_beentry() -
*
@@ -2656,9 +3086,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return &snapshot_archiverStats;
}
@@ -2673,9 +3104,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return &snapshot_globalStats;
}
@@ -2687,12 +3119,12 @@ pgstat_fetch_global(void)
* a pointer to the slru statistics struct.
* ---------
*/
-PgStat_SLRUStats *
+PgStat_StatSLRUEntry *
pgstat_fetch_slru(void)
{
- backend_read_statsfile();
+ pgstat_snapshot_global_stats();
- return slruStats;
+ return snapshot_SLRUStats;
}
@@ -2906,8 +3338,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3083,12 +3515,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3101,7 +3536,7 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
@@ -3118,6 +3553,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3378,7 +3815,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3673,9 +4111,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
break;
@@ -4310,94 +4745,71 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Report archiver statistics
* ----------
*/
void
-pgstat_send_archiver(const char *xlog, bool failed)
+pgstat_report_archiver(const char *xlog, bool failed)
{
- PgStat_MsgArchiver msg;
+ TimestampTz now = GetCurrentTimestamp();
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ if (failed)
+ {
+ /* Failed archival attempt */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ /* Successful archival operation */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *l = &BgWriterStats;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += l->timed_checkpoints;
+ shared_globalStats->requested_checkpoints += l->requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += l->checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += l->checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += l->buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += l->buf_written_clean;
+ shared_globalStats->maxwritten_clean += l->maxwritten_clean;
+ shared_globalStats->buf_written_backend += l->buf_written_backend;
+ shared_globalStats->buf_fsync_backend += l->buf_fsync_backend;
+ shared_globalStats->buf_alloc += l->buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4405,473 +4817,31 @@ pgstat_send_bgwriter(void)
MemSet(&BgWriterStats, 0, sizeof(BgWriterStats));
}
-/* ----------
- * pgstat_send_slru() -
- *
- * Send SLRU statistics to the collector
- * ----------
- */
-static void
-pgstat_send_slru(void)
-{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgSLRU all_zeroes;
-
- for (int i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /*
- * This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
- */
- if (memcmp(&SLRUStats[i], &all_zeroes, sizeof(PgStat_MsgSLRU)) == 0)
- continue;
-
- /* set the SLRU type before each send */
- SLRUStats[i].m_index = i;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&SLRUStats[i].m_hdr, PGSTAT_MTYPE_SLRU);
- pgstat_send(&SLRUStats[i], sizeof(PgStat_MsgSLRU));
-
- /*
- * Clear out the statistics buffer, so it can be re-used.
- */
- MemSet(&SLRUStats[i], 0, sizeof(PgStat_MsgSLRU));
- }
-}
-
-
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSLRUCOUNTER:
- pgstat_recv_resetslrucounter(&msg.msg_resetslrucounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_SLRU:
- pgstat_recv_slru(&msg.msg_slru, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
-{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
-
- return result;
-}
-
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
-{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->inserts_since_vacuum = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
-}
-
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
elog(DEBUG2, "writing stats file \"%s\"", statfile);
+ create_missing_dbentries();
+
/*
* Open the statistics temp file to write out the current values.
*/
@@ -4888,7 +4858,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4900,38 +4870,31 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Write SLRU stats struct
- */
- rc = fwrite(slruStats, sizeof(slruStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_DB, InvalidOid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &(*penv)->body;
+
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_database_stats(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
@@ -4942,6 +4905,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
+
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4974,55 +4939,19 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
}
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
+ * pgstat_write_database_stats() -
+ * Write the stat file for a single database.
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_database_stats(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -5030,8 +4959,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -5058,24 +4987,31 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_TABLE, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatTabEntry *tabentry = (PgStat_StatTabEntry *) &(*penv)->body;
+
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_FUNCTION, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatFuncEntry *funcentry =
+ (PgStat_StatFuncEntry *) &(*penv)->body;
+
fputc('F', fpout);
rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* No more output to be done. Close the temp file and replace the old
@@ -5109,102 +5045,165 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
+}
- if (permanent)
+/* ----------
+ * create_missing_dbentries() -
+ *
+ * There may be the case where database entry is missing for the database
+ * where object stats are recorded. This function creates such missing
+ * dbentries so that so that all stats entries can be written out to files.
+ * ----------
+ */
+static void
+create_missing_dbentries(void)
+{
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ HTAB *oidhash;
+ HASHCTL ctl;
+ HASH_SEQ_STATUS scan;
+ Oid *poid;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = CurrentMemoryContext;
+ oidhash = hash_create("Temporary table of OIDs",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Collect OID from the shared stats hash */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ hash_search(oidhash, &p->key.databaseid, HASH_ENTER, NULL);
+ dshash_seq_term(&hstat);
+
+ /* Create missing database entries if not exists. */
+ hash_seq_init(&scan, oidhash);
+ while ((poid = (Oid *) hash_seq_search(&scan)) != NULL)
+ (void) get_stat_entry(PGSTAT_TYPE_DB, *poid, InvalidOid,
+ false, init_dbentry, NULL);
+
+ hash_destroy(oidhash);
+}
+
+
+/* ----------
+ * get_stat_entry() -
+ *
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new envelope. If found is not NULL, it is set to true if
+ * existing entry is found or false if not.
+ * ----------
+ */
+static PgStatEnvelope *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait, entry_initializer initfunc, bool *found)
+{
+ bool create = (initfunc != NULL);
+ PgStatHashEntry *shent;
+ PgStatEnvelope *shenv = NULL;
+ PgStatHashEntryKey key;
+ bool myfound;
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ shent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &myfound);
+ if (shent)
{
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
+ if (create && !myfound)
+ {
+ /* Create new stats envelope. */
+ size_t envsize = PgStatEnvelopeSize(pgstat_entsize[type]);
+ dsa_pointer chunk = dsa_allocate0(area, envsize);
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
+ shenv = dsa_get_address(area, chunk);
+ shenv->type = type;
+ shenv->databaseid = dbid;
+ shenv->objectid = objid;
+ shenv->len = pgstat_entsize[type];
+ LWLockInitialize(&shenv->lock, LWTRANCHE_STATS);
+
+ /*
+ * The lock on dshsh is released just after. Call initializer
+ * callback before it is exposed to other process.
+ */
+ if (initfunc)
+ initfunc(shenv);
+
+ /* Link the new entry from the hash entry. */
+ shent->env = chunk;
+ }
+ else
+ shenv = dsa_get_address(area, shent->env);
+
+ dshash_release_lock(pgStatSharedHash, shent);
}
+
+ if (found)
+ *found = myfound;
+
+ return shenv;
}
+
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * Reads in existing activity statistics files into the shared stats hash.
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
+ PgStatEnvelope *env;
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
- int i;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
- memset(&slruStats, 0, sizeof(slruStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all SLRU items too.
- */
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- slruStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5213,7 +5212,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5221,49 +5220,30 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
/*
- * Read SLRU stats struct
- */
- if (fread(slruStats, 1, sizeof(slruStats), fpin) != sizeof(slruStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&slruStats, 0, sizeof(slruStats));
- goto done;
- }
-
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5277,7 +5257,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
fpin) != offsetof(PgStat_StatDBEntry, tables))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5286,76 +5266,33 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+
+ env = get_stat_entry(PGSTAT_TYPE_DB, dbbuf.databaseid,
+ InvalidOid,
+ false, init_dbentry, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ /* don't allow duplicate dbentries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, tables));
+ /* Read the data from the database-specific file. */
+ pgstat_read_db_statsfile(dbentry);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5365,59 +5302,50 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
/* ----------
* pgstat_read_db_statsfile() -
*
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry)
{
+ PgStatEnvelope *env;
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
FILE *fpin;
int32 format_id;
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5430,14 +5358,14 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5450,25 +5378,21 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE,
+ dbentry->databaseid, tabbuf.tableid,
+ false, init_tabentry, &found);
+ tabentry = (PgStat_StatTabEntry *) &env->body;
+ /* don't allow duplicate entries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5484,25 +5408,20 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE, dbentry->databaseid,
+ funcbuf.functionid,
+ false, init_funcentry, &found);
+ funcentry = (PgStat_StatFuncEntry *) &env->body;
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5518,7 +5437,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5526,304 +5445,15 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read SLRU stats struct
- */
- if (fread(mySLRUStats, 1, sizeof(mySLRUStats), fpin) != sizeof(mySLRUStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
-done:
FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
-}
-
-/* ----------
- * pgstat_setup_memcxt() -
- *
- * Create pgStatLocalContext, if not already done.
- * ----------
- */
-static void
-pgstat_setup_memcxt(void)
-{
- if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
@@ -5842,797 +5472,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum = tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- tabentry->inserts_since_vacuum = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum += tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
}
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
+ if (pgStatSnapshotContext)
{
- char statfile[MAXPGPATH];
+ MemoryContextReset(pgStatSnapshotContext);
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
+ /* Reset variables that pointed to the context */
+ global_snapshot_is_valid = false;
+ pgStatSnapshotHash = NULL;
}
}
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_resetslrucounter() -
- *
- * Reset some SLRU statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len)
-{
- int i;
- TimestampTz ts = GetCurrentTimestamp();
-
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /* reset entry with the given index, or all entries (index is -1) */
- if ((msg->m_index == -1) || (msg->m_index == i))
- {
- memset(&slruStats[i], 0, sizeof(slruStats[i]));
- slruStats[i].stat_reset_timestamp = ts;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signaling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * It is quite possible that a non-aggressive VACUUM ended up skipping
- * various pages, however, we'll zero the insert counter here regardless.
- * It's currently used only to track when we need to perform an "insert"
- * autovacuum, which are mainly intended to freeze newly inserted tuples.
- * Zeroing this may just mean we'll not try to vacuum the table again
- * until enough tuples have been inserted to trigger another insert
- * autovacuum. An anti-wraparound autovacuum will catch any persistent
- * stragglers.
- */
- tabentry->inserts_since_vacuum = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_slru() -
- *
- * Process a SLRU message.
- * ----------
- */
-static void
-pgstat_recv_slru(PgStat_MsgSLRU *msg, int len)
-{
- slruStats[msg->m_index].blocks_zeroed += msg->m_blocks_zeroed;
- slruStats[msg->m_index].blocks_hit += msg->m_blocks_hit;
- slruStats[msg->m_index].blocks_read += msg->m_blocks_read;
- slruStats[msg->m_index].blocks_written += msg->m_blocks_written;
- slruStats[msg->m_index].blocks_exists += msg->m_blocks_exists;
- slruStats[msg->m_index].flush += msg->m_flush;
- slruStats[msg->m_index].truncate += msg->m_truncate;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
-}
-
/*
* Convert a potentially unsafely truncated activity string (see
* PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
@@ -6721,7 +5579,7 @@ pgstat_slru_name(int slru_idx)
* Returns pointer to entry with counters for given SLRU (based on the name
* stored in SlruCtl as lwlock tranche name).
*/
-static inline PgStat_MsgSLRU *
+static PgStat_StatSLRUEntry *
slru_entry(int slru_idx)
{
/*
@@ -6732,7 +5590,7 @@ slru_entry(int slru_idx)
Assert((slru_idx >= 0) && (slru_idx < SLRU_NUM_ELEMENTS));
- return &SLRUStats[slru_idx];
+ return &local_SLRUStats[slru_idx];
}
/*
@@ -6742,41 +5600,48 @@ slru_entry(int slru_idx)
void
pgstat_count_slru_page_zeroed(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_zeroed += 1;
+ slru_entry(slru_idx)->blocks_zeroed += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_hit(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_hit += 1;
+ slru_entry(slru_idx)->blocks_hit += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_exists(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_exists += 1;
+ slru_entry(slru_idx)->blocks_exists += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_read(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_read += 1;
+ slru_entry(slru_idx)->blocks_read += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_written(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_written += 1;
+ slru_entry(slru_idx)->blocks_written += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_flush(int slru_idx)
{
- slru_entry(slru_idx)->m_flush += 1;
+ slru_entry(slru_idx)->flush += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_truncate(int slru_idx)
{
- slru_entry(slru_idx)->m_truncate += 1;
+ slru_entry(slru_idx)->truncate += 1;
+ have_slrustats = true;
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 30f3fc6ba1..6b7baa4ebf 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -254,7 +254,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -502,7 +501,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1325,12 +1323,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1779,11 +1771,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2706,8 +2693,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3070,8 +3055,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3138,13 +3121,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3217,22 +3193,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3693,22 +3653,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3904,8 +3848,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3929,8 +3871,7 @@ PostmasterStateMachine(void)
{
/*
* PM_WAIT_DEAD_END state ends when the BackendList is entirely empty
- * (ie, no dead_end children remain), and the archiver and stats
- * collector are gone too.
+ * (ie, no dead_end children remain), and the archiveris gone too.
*
* The reason we wait for those two is to protect them against a new
* postmaster starting conflicting subprocesses; this isn't an
@@ -3940,8 +3881,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4142,8 +4082,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5116,18 +5054,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5246,12 +5172,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6146,7 +6066,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6202,8 +6121,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6438,7 +6355,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 3b46bfe9ab..b6c7a8bc3c 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1550,8 +1550,8 @@ is_checksummed_file(const char *fullpath, const char *filename)
*
* If 'missing_ok' is true, will not throw an error if the file is not found.
*
- * If dboid is anything other than InvalidOid then any checksum failures detected
- * will get reported to the stats collector.
+ * If dboid is anything other than InvalidOid then any checksum failures
+ * detected will get reported to the activity stats facility.
*
* Returns true if the file was successfully sent, false if 'missing_ok',
* and the file did not exist.
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 29c920800a..32c511542d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2010,7 +2010,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ BgWriterStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2120,7 +2120,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2310,7 +2310,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2318,7 +2318,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 427b0d59cd..58a442f482 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 2fa90cc095..17a46db74b 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -177,7 +177,9 @@ static const char *const BuiltinTrancheNames[] = {
/* LWTRANCHE_PARALLEL_APPEND: */
"ParallelAppend",
/* LWTRANCHE_PER_XACT_PREDICATE_LIST: */
- "PerXactPredicateList"
+ "PerXactPredicateList",
+ /* LWTRANCHE_STATS: */
+ "ActivityStatistics"
};
StaticAssertDecl(lengthof(BuiltinTrancheNames) ==
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index e6985e8eed..cdd7095ebf 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -50,3 +50,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
LogicalRepWorkerLock 43
XactTruncationLock 44
+StatsLock 45
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 7d667c6586..3535089755 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -414,8 +414,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
DropRelFileNodesAllBuffers(rnodes, nrels);
/*
- * It'd be nice to tell the stats collector to forget them immediately,
- * too. But we can't because we don't know the OIDs.
+ * It'd be nice to tell the activity stats facility to forget them
+ * immediately, too. But we can't because we don't know the OIDs.
*/
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..0396aa469f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3209,6 +3209,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3783,6 +3789,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4185,11 +4192,12 @@ PostgresMain(int argc, char *argv[],
* Note: this includes fflush()'ing the last of the prior output.
*
* This is also a good time to send collected statistics to the
- * collector, and to update the PS stats display. We avoid doing
- * those every time through the message loop because it'd slow down
- * processing of batched messages, and because we don't want to report
- * uncommitted updates (that confuses autovacuum). The notification
- * processor wants a call too, if we are not in a transaction block.
+ * activity statistics, and to update the PS stats display. We avoid
+ * doing those every time through the message loop because it'd slow
+ * down processing of batched messages, and because we don't want to
+ * report uncommitted updates (that confuses autovacuum). The
+ * notification processor wants a call too, if we are not in a
+ * transaction block.
*/
if (send_ready_for_query)
{
@@ -4221,6 +4229,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4233,8 +4243,13 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4269,7 +4284,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4277,6 +4292,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 2aff739466..b9e78dd8e8 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1,7 +1,7 @@
/*-------------------------------------------------------------------------
*
* pgstatfuncs.c
- * Functions for accessing the statistics collector data
+ * Functions for accessing the activity statistics data
*
* Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -35,9 +35,6 @@
#define HAS_PGSTAT_PERMISSIONS(role) (is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS) || has_privs_of_role(GetUserId(), role))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1261,7 +1258,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1277,7 +1274,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1293,7 +1290,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1309,7 +1306,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1325,7 +1322,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1341,7 +1338,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1357,7 +1354,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1373,7 +1370,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1389,7 +1386,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1422,7 +1419,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1438,7 +1435,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1453,7 +1450,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1468,7 +1465,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1483,7 +1480,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1498,7 +1495,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1513,7 +1510,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1528,11 +1525,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1547,7 +1544,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1565,7 +1562,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1602,7 +1599,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1618,7 +1615,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1704,7 +1701,7 @@ pg_stat_get_slru(PG_FUNCTION_ARGS)
MemoryContext per_query_ctx;
MemoryContext oldcontext;
int i;
- PgStat_SLRUStats *stats;
+ PgStat_StatSLRUEntry *stats;
/* check to see if caller supports us returning a tuplestore */
if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
@@ -1738,7 +1735,7 @@ pg_stat_get_slru(PG_FUNCTION_ARGS)
/* for each row */
Datum values[PG_STAT_GET_SLRU_COLS];
bool nulls[PG_STAT_GET_SLRU_COLS];
- PgStat_SLRUStats stat = stats[i];
+ PgStat_StatSLRUEntry stat = stats[i];
const char *name;
name = pgstat_slru_name(i);
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..51748c99ad 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index cca9704d2d..430e997ca6 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -241,9 +241,6 @@ GetBackendTypeDesc(BackendType backendType)
case B_ARCHIVER:
backendDesc = "archiver";
break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
case B_LOGGER:
backendDesc = "logger";
break;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index f4247ea70d..f65d05c24c 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -631,6 +632,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1241,6 +1244,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 2f3e0a70e0..f073025f69 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -747,8 +747,8 @@ const char *const config_group_names[] =
gettext_noop("Statistics"),
/* STATS_MONITORING */
gettext_noop("Statistics / Monitoring"),
- /* STATS_COLLECTOR */
- gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* STATS_ACTIVITY */
+ gettext_noop("Statistics / Query and Index Activity Statistics"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
@@ -1478,7 +1478,7 @@ static struct config_bool ConfigureNamesBool[] =
#endif
{
- {"track_activities", PGC_SUSET, STATS_COLLECTOR,
+ {"track_activities", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects information about executing commands."),
gettext_noop("Enables the collection of information on the currently "
"executing command of each session, along with "
@@ -1489,7 +1489,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_counts", PGC_SUSET, STATS_COLLECTOR,
+ {"track_counts", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects statistics on database activity."),
NULL
},
@@ -1498,7 +1498,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_io_timing", PGC_SUSET, STATS_COLLECTOR,
+ {"track_io_timing", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects timing statistics for database I/O activity."),
NULL
},
@@ -4300,7 +4300,7 @@ static struct config_string ConfigureNamesString[] =
},
{
- {"stats_temp_directory", PGC_SIGHUP, STATS_COLLECTOR,
+ {"stats_temp_directory", PGC_SIGHUP, STATS_ACTIVITY,
gettext_noop("Writes temporary statistics files to the specified directory."),
NULL,
GUC_SUPERUSER_ONLY
@@ -4636,7 +4636,7 @@ static struct config_enum ConfigureNamesEnum[] =
},
{
- {"track_functions", PGC_SUSET, STATS_COLLECTOR,
+ {"track_functions", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects function-level statistics on database activity."),
NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ac02bd0c00..e813af7676 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -574,7 +574,7 @@
# STATISTICS
#------------------------------------------------------------------------------
-# - Query and Index Statistics Collector -
+# - Query and Index Activity Statistics -
#track_activities = on
#track_counts = on
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 208df557b8..4d07829ecd 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -124,7 +124,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index cf0b463eba..de7baafb88 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,8 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -320,7 +322,6 @@ typedef enum BackendType
B_WAL_SENDER,
B_WAL_WRITER,
B_ARCHIVER,
- B_STATS_COLLECTOR,
B_LOGGER,
} BackendType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c55dc1481c..eee9feb8f7 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -15,9 +15,11 @@
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,35 +43,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_RESETSLRUCOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_SLRU,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -80,9 +53,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -113,18 +85,17 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_delta_live_tuples;
PgStat_Counter t_delta_dead_tuples;
PgStat_Counter t_changed_tuples;
+ bool reset_changed_tuples;
PgStat_Counter t_blocks_fetched;
PgStat_Counter t_blocks_hit;
+
+ PgStat_Counter vacuum_count;
+ PgStat_Counter autovac_vacuum_count;
+ PgStat_Counter analyze_count;
+ PgStat_Counter autovac_analyze_count;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -158,6 +129,10 @@ typedef struct PgStat_TableStatus
Oid t_id; /* table's OID */
bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
} PgStat_TableStatus;
@@ -183,308 +158,32 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgResetslrucounter Sent by the backend to tell the collector
- * to reset a SLRU counter
- * ----------
- */
-typedef struct PgStat_MsgResetslrucounter
-{
- PgStat_MsgHdr m_hdr;
- int m_index;
-} PgStat_MsgResetslrucounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgSLRU Sent by a backend to update SLRU statistics.
- * ----------
- */
-typedef struct PgStat_MsgSLRU
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_index;
- PgStat_Counter m_blocks_zeroed;
- PgStat_Counter m_blocks_hit;
- PgStat_Counter m_blocks_read;
- PgStat_Counter m_blocks_written;
- PgStat_Counter m_blocks_exists;
- PgStat_Counter m_flush;
- PgStat_Counter m_truncate;
-} PgStat_MsgSLRU;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgTempFile
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter buf_alloc;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -516,98 +215,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgResetslrucounter msg_resetslrucounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgSLRU msg_slru;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -616,13 +225,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9D
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -632,7 +237,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -642,29 +246,72 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ Oid databaseid;
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/*
+ * SLRU statistics kept in the stats collector
+ */
+typedef struct PgStat_StatSLRUEntry
+{
+ PgStat_Counter blocks_zeroed;
+ PgStat_Counter blocks_hit;
+ PgStat_Counter blocks_read;
+ PgStat_Counter blocks_written;
+ PgStat_Counter blocks_exists;
+ PgStat_Counter flush;
+ PgStat_Counter truncate;
+ TimestampTz stat_reset_timestamp;
+} PgStat_StatSLRUEntry;
+
+
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_HashMountInfo dshash mount information
+ * ----------
+ */
+typedef struct PgStat_HashMountInfo
+{
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
+} PgStat_HashMountInfo;
+
+/* ----------
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
Oid tableid;
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -684,19 +331,15 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -709,9 +352,8 @@ typedef struct PgStat_StatFuncEntry
PgStat_Counter f_self_time;
} PgStat_StatFuncEntry;
-
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -727,7 +369,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -745,21 +387,6 @@ typedef struct PgStat_GlobalStats
TimestampTz stat_reset_timestamp;
} PgStat_GlobalStats;
-/*
- * SLRU statistics kept in the stats collector
- */
-typedef struct PgStat_SLRUStats
-{
- PgStat_Counter blocks_zeroed;
- PgStat_Counter blocks_hit;
- PgStat_Counter blocks_read;
- PgStat_Counter blocks_written;
- PgStat_Counter blocks_exists;
- PgStat_Counter flush;
- PgStat_Counter truncate;
- TimestampTz stat_reset_timestamp;
-} PgStat_SLRUStats;
-
/* ----------
* Backend states
@@ -808,7 +435,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1054,7 +680,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1251,13 +877,15 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1272,29 +900,26 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
@@ -1456,8 +1081,8 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1466,13 +1091,16 @@ extern void pgstat_send_bgwriter(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_snapshot(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
-extern PgStat_SLRUStats *pgstat_fetch_slru(void);
+extern PgStat_StatSLRUEntry *pgstat_fetch_slru(void);
extern void pgstat_count_slru_page_zeroed(int slru_idx);
extern void pgstat_count_slru_page_hit(int slru_idx);
@@ -1483,5 +1111,6 @@ extern void pgstat_count_slru_flush(int slru_idx);
extern void pgstat_count_slru_truncate(int slru_idx);
extern const char *pgstat_slru_name(int slru_idx);
extern int pgstat_slru_index(const char *name);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index c04ae97148..ce2b79d1bc 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -219,6 +219,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_SHARED_TIDBITMAP,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_PER_XACT_PREDICATE_LIST,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index 454c2df487..2c65231b04 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -88,7 +88,7 @@ enum config_group
PROCESS_TITLE,
STATS,
STATS_MONITORING,
- STATS_COLLECTOR,
+ STATS_ACTIVITY,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 83a15f6795..77d1572a99 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.18.2
* [PATCH v29 5/7] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 500ms and such large data is
serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 500ms. Locks
on the shared statistics is acquired by the units of such like tables,
functions so the expected chance of collision are not so
high. Furthermore, until 1 second has elapsed since the last flushing
to shared stats, lock failure postpones stats flushing so that lock
contention doesn't slow down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/transam/xlog.c | 4 +-
src/backend/catalog/index.c | 24 +-
src/backend/postmaster/autovacuum.c | 54 +-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 12 +-
src/backend/postmaster/pgarch.c | 4 +-
src/backend/postmaster/pgstat.c | 4865 +++++++-----------
src/backend/postmaster/postmaster.c | 85 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 1 +
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/tcop/postgres.c | 26 +-
src/backend/utils/adt/pgstatfuncs.c | 53 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 +
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 2 +
src/include/pgstat.h | 514 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/timeout.h | 1 +
21 files changed, 2063 insertions(+), 3612 deletions(-)
inline patch 1: src/backend/access/transam/xlog.c, src/backend/catalog/index.c, src/backend/postmaster/autovacuum.c and 18 more (+2063/-3612 lines) download inline-1.patch
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index d75a257da2..99beaf0215 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8594,9 +8594,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bd7ec923e9..31202766e2 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1699,28 +1699,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be sent by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Close relations */
table_close(pg_class, RowExclusiveLock);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7e97ffab27..800693af4a 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -338,9 +338,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1938,8 +1935,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1958,12 +1953,6 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
@@ -2011,9 +2000,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2092,8 +2078,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2176,8 +2162,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2736,29 +2722,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2779,17 +2742,12 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
bool doanalyze;
autovac_table *tab = NULL;
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
bool wraparound;
AutoVacOpts *avopts;
/* use fresh stats */
autovac_refresh_stats();
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* fetch the relation's relcache entry */
classTup = SearchSysCacheCopy1(RELOID, ObjectIdGetDatum(relid));
if (!HeapTupleIsValid(classTup))
@@ -2813,8 +2771,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
}
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..94bdd664b5 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -236,7 +236,7 @@ BackgroundWriterMain(void)
/*
* Send off activity statistics to the stats collector
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..8a2fd0ddb2 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -350,7 +350,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ BgWriterStats.requested_checkpoints++;
}
/*
@@ -364,7 +364,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ BgWriterStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -492,7 +492,7 @@ CheckpointerMain(void)
* worth the trouble to split the stats support into two independent
* stats message types.)
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* Sleep until we are signaled or it's time for another checkpoint or
@@ -693,7 +693,7 @@ CheckpointWriteDelay(int flags, double progress)
/*
* Report interim activity statistics to the stats collector.
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1238,8 +1238,8 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ BgWriterStats.buf_written_backend += CheckpointerShmem->num_backend_writes;
+ BgWriterStats.buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 6fe7a136ba..f0b524ca50 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -401,7 +401,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we successfully
* archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
@@ -411,7 +411,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we failed to
* archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index ab42df7e1b..7679b20833 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (1000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (250ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (10000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,68 +35,43 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 1000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 250 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 10000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -116,7 +86,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -131,75 +100,163 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-
-/* ----------
- * Local data
- * ----------
+ * Shared stats bootstrap information, protected by StatsLock.
+ *
+ * refcount is used to know whether a process going to detach shared stats is
+ * the last process or not. The last process writes out the stats files.
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ dsa_pointer global_stats; /* DSA pointer to global stats */
+ dsa_pointer archiver_stats; /* Ditto for archiver stats */
+ int refcount; /* # of processes that is attaching the shared
+ * stats memory */
+} StatsShmemStruct;
-static time_t last_pgstat_start_time;
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
+ * Enums and types to define shared statistics structure.
+ *
+ * Statistics entries for each object is stored in individual DSA-allocated
+ * memory. Every entry is pointed from the dshash pgStatSharedHash via
+ * dsa_pointer. The structure makes object-stats entries not moved by dshash
+ * resizing, and allows the dshash can release lock sooner on stats
+ * updates. Also it reduces interfering among write-locks on each stat entry by
+ * not relying on partition lock of dshash. PgStatLocalHashEntry is the
+ * local-stats equivalent of PgStatHashEntry for shared stat entries.
+ *
+ * Each stat entry is enveloped by the type PgStatEnvelope, which stores common
+ * attribute of all kind of statistics and a LWLock lock object.
*
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStatEnvelope (dsa memory block)
+ *
+ * Local stats are stored as:
+ *
+ * dshash pgStatLocalHash
+ * -> PgStatLocalHashEntry (dynahash entry)
+ * (direct pointer)-> PgStatEnvelope (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries. */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_ALL, /* Not a type, for the parameters of
+ * pgstat_collect_stat_entries */
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION, /* per-function statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static size_t pgstat_entsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry) /* PGSTAT_TYPE_FUNCTION */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static size_t pgstat_localentsize[] =
+{
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry) /* PGSTAT_TYPE_FUNCTION */
+};
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashEntryKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object OID */
+} PgStatHashEntryKey;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Stats numbers that are waiting for flushing out to shared stats are held in
+ * pgStatLocalHash,
*/
-static HTAB *pgStatFunctions = NULL;
+typedef struct PgStatHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ dsa_pointer env; /* pointer to shared stats envelope in DSA */
+} PgStatHashEntry;
+
+/* struct for shared statistics entry pointed from shared hash entry. */
+typedef struct PgStatEnvelope
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* databaseid */
+ Oid objectid; /* objectid */
+ size_t len; /* length of body, fixed per type. */
+ LWLock lock; /* lightweight lock to protect body */
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatEnvelope;
+
+#define PgStatEnvelopeSize(bodylen) \
+ (offsetof(PgStatEnvelope, body) + (bodylen))
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ PgStatEnvelope *env; /* pointer to stats envelope in heap */
+} PgStatLocalHashEntry;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * Snapshot is stats entry that is locally copied to offset stable values for a
+ * transaction.
*/
-static bool have_function_stats = false;
+typedef struct PgStatSnapshot
+{
+ PgStatHashEntryKey key;
+ bool negative;
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatSnapshot;
+
+#define PgStatSnapshotSize(bodylen) \
+ (offsetof(PgStatSnapshot, body) + (bodylen))
+
+/* parameter for shared hashes */
+static const dshash_parameters dsh_rootparams = {
+ sizeof(PgStatHashEntryKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
+
+/* Local stats numbers are stored here */
+static HTAB *pgStatLocalHash = NULL;
+
+#define HAVE_ANY_PENDING_STATS() \
+ (pgStatLocalHash != NULL || \
+ pgStatXactCommit != 0 || pgStatXactRollback != 0)
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -236,11 +293,10 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatSnapshotHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -249,19 +305,17 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are collected not per database nor per table
+ * basis. shared_* points to shared memory and snapshot_* are backend
+ * snapshots.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+static bool global_snapshot_is_valid = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats snapshot_globalStats;
/*
* Total time charged to functions so far in the current backend.
@@ -270,523 +324,269 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/*
+ * Newly created shared stats entries needs to be initialized before the other
+ * processes get access it. get_stat_entry() calls it for the purpose.
+ */
+typedef void (*entry_initializer) (PgStatEnvelope * env);
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStatEnvelope * get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait,
+ entry_initializer initfunc, bool *found);
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid);
+static void create_missing_dbentries(void);
+static void pgstat_write_database_stats(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry);
+
+static bool flush_tabstat(PgStatEnvelope * env, bool nowait);
+static bool flush_funcstat(PgStatEnvelope * env, bool nowait);
+static bool flush_dbstat(PgStatEnvelope * env, bool nowait);
+
+static void init_dbentry(PgStatEnvelope * env);
+static void init_funcentry(PgStatEnvelope * env);
+static void init_tabentry(PgStatEnvelope * env);
+
+static bool delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait);
+static PgStatEnvelope * get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static void pgstat_snapshot_global_stats(void);
+
static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
-
-static void pgstat_setup_memcxt(void);
-
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
+ bool found;
-#define TESTBYTEVAL ((char) 199)
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
+}
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgStatLocalContext and pgStatSnapshotContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgStatLocalContext)
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+
+ if (!pgStatSnapshotContext)
+ pgStatSnapshotContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Database statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+}
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read saved statistics files if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * Don't use dsm under postmaster, or when not tracking counts.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ pgstat_setup_memcxt();
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ if (area)
+ return;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
+ /*
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that processes going to exit can find whether it
+ * is the last or not.
+ */
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
+ {
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_rootparams, 0);
- test_byte++; /* just make sure variable is changed */
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Load saved data if any. */
+ pgstat_read_statsfiles();
- /* If we get here, we have a working socket */
- break;
+ StatsShmem->refcount = 1;
}
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockRelease(StatsLock);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * If we're not the first process, attach existing shared stats area
+ * outside the StatsLock section.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (!area)
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* Attach shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_rootparams,
+ StatsShmem->hash_handle, 0);
+
+ /* Setup local variables */
+ pgStatLocalHash = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
}
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
- }
-
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+ global_snapshot_is_valid = false;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_stats)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
- {
- int nchars;
- Oid tmp_oid;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount < 1)
+ {
/*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
+ * The process is the last one that is attaching the shared stats
+ * memory. Write out the stats files if requested.
*/
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
+ if (write_stats)
+ pgstat_write_statsfiles();
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ /* No one is using the area. */
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
- FreeDir(dir);
+
+ LWLockRelease(StatsLock);
+
+ /*
+ * Detach the area. Automatically destroyed when the last process detached
+ * it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatLocalHash = NULL;
+ global_snapshot_is_valid = false;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
-#ifdef EXEC_BACKEND
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
-
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
/*
- * Okay, fork off the collector.
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
+ detach_shared_stats(false); /* Don't write files. */
+ attach_shared_stats();
}
/* ------------------------------------------------------------
@@ -794,144 +594,488 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait = !force; /* Don't use force ever after */
+ HASH_SEQ_STATUS scan;
+ PgStatLocalHashEntry *lent;
+ PgStatLocalHashEntry **dbentlist;
+ int dbentlistlen = 8;
+ int ndbentries = 0;
+ int remains = 0;
int i;
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (area == NULL || !HAVE_ANY_PENDING_STATS())
+ return 0;
+
+ dbentlist = palloc(sizeof(PgStatLocalHashEntry *) * dbentlistlen);
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
+ if (nowait)
+ {
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
+ {
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ nowait = false;
+ }
/*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * flush_tabstat applies some of stats numbers of flushed entries into
+ * local database stats. So flush-out database stats later.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ if (pgStatLocalHash)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /* Step 1: flush out other than database stats */
+ hash_seq_init(&scan, pgStatLocalHash);
+ while ((lent = (PgStatLocalHashEntry *) hash_seq_search(&scan)) != NULL)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
+ bool remove = false;
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
+ switch (lent->env->type)
+ {
+ case PGSTAT_TYPE_DB:
+ if (ndbentries >= dbentlistlen)
+ {
+ dbentlistlen *= 2;
+ dbentlist = repalloc(dbentlist,
+ sizeof(PgStatLocalHashEntry *) *
+ dbentlistlen);
+ }
+ dbentlist[ndbentries++] = lent;
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent->env, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent->env, nowait))
+ remove = true;
+ break;
+ default:
+ Assert(false);
+ }
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
+ }
+
+ /* Step 2: flush out database stats */
+ for (i = 0; i < ndbentries; i++)
+ {
+ PgStatLocalHashEntry *lent = dbentlist[i];
+
+ if (flush_dbstat(lent->env, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ pfree(dbentlist);
+
+ if (remains <= 0)
+ {
+ hash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
+ }
+
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (shared_globalStats->stats_timestamp < now)
+ shared_globalStats->stats_timestamp = now;
+ LWLockRelease(StatsLock);
+
+ /*
+ * If we have pending local stats, let the caller know the retry interval.
+ */
+ if (HAVE_ANY_PENDING_STATS())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
+
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
+
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
+
+ return retry_interval / 1000;
}
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
+}
+
+/*
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_tabstat(PgStatEnvelope * lenv, bool nowait)
+{
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+ bool found;
+
+ Assert(lenv->type == PGSTAT_TYPE_TABLE);
+
+ lstats = (PgStat_TableStatus *) &lenv->body;
+ dboid = lstats->t_shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_TABLE, dboid, lstats->t_id,
+ nowait, init_tabentry, &found);
+
+ /* skip if dshash failed to acquire lock */
+ if (shenv == NULL)
+ return false;
+
+ /* retrieve the shared table stats entry from the envelope */
+ shtabstats = (PgStat_StatTabEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated ||
+ lstats->t_counts.vacuum_count > 0 ||
+ lstats->t_counts.analyze_count > 0 ||
+ lstats->t_counts.autovac_vacuum_count > 0 ||
+ lstats->t_counts.autovac_analyze_count > 0)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ /* clear the change counter if requested */
+ if (lstats->t_counts.reset_changed_tuples)
+ shtabstats->changes_since_analyze = 0;
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /*
+ * Update vacuum/analyze timestamp and counters, so that the values won't
+ * goes back.
+ */
+ if (shtabstats->vacuum_timestamp < lstats->vacuum_timestamp)
+ shtabstats->vacuum_timestamp = lstats->vacuum_timestamp;
+ shtabstats->vacuum_count += lstats->t_counts.vacuum_count;
+
+ if (shtabstats->autovac_vacuum_timestamp < lstats->autovac_vacuum_timestamp)
+ shtabstats->autovac_vacuum_timestamp = lstats->autovac_vacuum_timestamp;
+ shtabstats->autovac_vacuum_count += lstats->t_counts.autovac_vacuum_count;
+
+ if (shtabstats->analyze_timestamp < lstats->analyze_timestamp)
+ shtabstats->analyze_timestamp = lstats->analyze_timestamp;
+ shtabstats->analyze_count += lstats->t_counts.analyze_count;
+
+ if (shtabstats->autovac_analyze_timestamp < lstats->autovac_analyze_timestamp)
+ shtabstats->autovac_analyze_timestamp = lstats->autovac_analyze_timestamp;
+ shtabstats->autovac_analyze_count += lstats->t_counts.autovac_analyze_count;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shenv->lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ return true;
+}
+
+/* ----------
+ * init_tabentry() -
+ *
+ * initializes table stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_tabentry(PgStatEnvelope * env)
+{
+ PgStat_StatTabEntry *tabent = (PgStat_StatTabEntry *) &env->body;
+
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * If it's a new table entry, initialize counters to the values we just
+ * got.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
-
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ Assert(env->type == PGSTAT_TYPE_TABLE);
+ tabent->tableid = env->objectid;
+ tabent->numscans = 0;
+ tabent->tuples_returned = 0;
+ tabent->tuples_fetched = 0;
+ tabent->tuples_inserted = 0;
+ tabent->tuples_updated = 0;
+ tabent->tuples_deleted = 0;
+ tabent->tuples_hot_updated = 0;
+ tabent->n_live_tuples = 0;
+ tabent->n_dead_tuples = 0;
+ tabent->changes_since_analyze = 0;
+ tabent->blocks_fetched = 0;
+ tabent->blocks_hit = 0;
+
+ tabent->vacuum_timestamp = 0;
+ tabent->vacuum_count = 0;
+ tabent->autovac_vacuum_timestamp = 0;
+ tabent->autovac_vacuum_count = 0;
+ tabent->analyze_timestamp = 0;
+ tabent->analyze_count = 0;
+ tabent->autovac_analyze_timestamp = 0;
+ tabent->autovac_analyze_count = 0;
}
+
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * flush_funcstat - flush out a local function stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatEnvelope * env, bool nowait)
+{
+ /* we assume this inits to all zeroes: */
+ static const PgStat_FunctionCounts all_zeroes;
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatFuncEntry *sharedent = NULL; /* shared stats entry */
+ bool found;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) &env->body;
+
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&localent->f_counts, &all_zeroes,
+ sizeof(PgStat_FunctionCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, localent->f_id,
+ nowait, init_funcentry, &found);
+ /* skip if dshash failed to acquire lock */
+ if (sharedent == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* retrieve the shared table stats entry from the envelope */
+ sharedent = (PgStat_StatFuncEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->f_numcalls += localent->f_counts.f_numcalls;
+ sharedent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ sharedent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shenv->lock);
+
+ return true;
+}
+
+
+/* ----------
+ * init_funcentry() -
+ *
+ * initializes function stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
*/
static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+init_funcentry(PgStatEnvelope * env)
{
- int n;
- int len;
+ PgStat_StatFuncEntry *shstat = (PgStat_StatFuncEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ shstat->functionid = env->objectid;
+ shstat->f_numcalls = 0;
+ shstat->f_total_time = 0;
+ shstat->f_self_time = 0;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatEnvelope * env, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStatEnvelope *shenv;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &env->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_DB, localent->databaseid, InvalidOid,
+ nowait, init_dbentry, NULL);
+
+ /* skip if dshash failed to acquire lock */
+ if (!shenv)
+ return false;
+
+ /* retrieve the shared stats entry from the envelope */
+ sharedent = (PgStat_StatDBEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(localent->databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -939,257 +1083,102 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&shenv->lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
+
+/* ----------
+ * init_dbentry() -
+ *
+ * initializes database stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_dbentry(PgStatEnvelope * env)
+{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+ dbentry->databaseid = env->databaseid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ /* initialize the new shared entry */
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
+}
+
+
/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+ * Create the filename for a DB stat file; filename is output parameter points
+ * to a character buffer of length len.
*/
static void
-pgstat_send_funcstats(void)
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
- {
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
-
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
}
/* ----------
- * pgstat_vacuum_stat() -
+ * collect_stat_entries() -
*
- * Will tell the collector about objects he can get rid of.
+ * Collect the shared statistics entries specified by type and dbid. Returns a
+ * list of pointer to shared statistics in palloc'ed memory. If type is
+ * PGSTAT_TYPE_ALL, all types of statistics of the database is collected. If
+ * type is PGSTAT_TYPE_DB, the parameter dbid is ignored and collect all
+ * PGSTAT_TYPE_DB entries.
* ----------
*/
-void
-pgstat_vacuum_stat(void)
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
- return;
-
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
-
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ int listlen = 16;
+ PgStatEnvelope **envlist = palloc(sizeof(PgStatEnvelope * *) * listlen);
+ int n = 0;
+
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ if ((type != PGSTAT_TYPE_ALL && p->key.type != type) ||
+ (type != PGSTAT_TYPE_DB && p->key.databaseid != dbid))
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
+ if (n >= listlen - 1)
{
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
+ listlen *= 2;
+ envlist = repalloc(envlist, listlen * sizeof(PgStatEnvelope * *));
}
+ envlist[n++] = dsa_get_address(area, p->env);
}
+ dshash_seq_term(&hstat);
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
+ envlist[n] = NULL;
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
- }
+ return envlist;
}
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
* into a temporary hash table. Caller should hash_destroy the result
@@ -1198,7 +1187,7 @@ pgstat_vacuum_stat(void)
* ----------
*/
static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
HTAB *htab;
HASHCTL hash_ctl;
@@ -1212,7 +1201,7 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
hash_ctl.entrysize = sizeof(Oid);
hash_ctl.hcxt = CurrentMemoryContext;
htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
+ PGSTAT_TABLE_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
@@ -1239,65 +1228,184 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
}
+/* ----------
+ * pgstat_vacuum_stat() -
+ *
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
+ * ----------
+ */
+void
+pgstat_vacuum_stat(void)
+{
+ HTAB *dbids; /* database ids */
+ HTAB *relids; /* relation ids in the current database */
+ HTAB *funcids; /* function ids in the current database */
+ PgStatEnvelope **victims; /* victim entry list */
+ int arraylen = 0; /* storage size of the above */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+ int i;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+
+ /* collect victims from shared stats */
+ arraylen = 16;
+ victims = palloc(sizeof(PgStatEnvelope * *) * arraylen);
+ nvictims = 0;
+
+ dshash_seq_init(&dshstat, pgStatSharedHash, false);
+
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ HTAB *oidtab;
+ Oid *key;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
+ continue;
+
+ switch (ent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidtab = dbids;
+ key = &ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidtab = relids;
+ key = &ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidtab = funcids;
+ key = &ent->key.objectid;
+ break;
+ case PGSTAT_TYPE_ALL:
+ Assert(false);
+ break;
+ }
+
+ /* Skip existent objects. */
+ if (hash_search(oidtab, key, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* extend the list if needed */
+ if (nvictims >= arraylen)
+ {
+ arraylen *= 2;
+ victims = repalloc(victims, sizeof(PgStatEnvelope * *) * arraylen);
+ }
+
+ victims[nvictims++] = dsa_get_address(area, ent->env);
+ }
+ dshash_seq_term(&dshstat);
+ hash_destroy(dbids);
+ hash_destroy(relids);
+ hash_destroy(funcids);
+
+ /* Now try removing the victim entries */
+ for (i = 0; i < nvictims; i++)
+ {
+ PgStatEnvelope *p = victims[i];
+
+ delete_stat_entry(p->type, p->databaseid, p->objectid, true);
+ }
+}
+
+
+/* ----------
+ * delete_stat_entry() -
+ *
+ * Deletes the specified entry from shared stats hash.
+ *
+ * Returns true when successfully deleted.
+ * ----------
+ */
+static bool
+delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait)
+{
+ PgStatHashEntryKey key;
+ PgStatHashEntry *ent;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ ent = dshash_find_extended(pgStatSharedHash, &key,
+ true, nowait, false, NULL);
+
+ if (!ent)
+ return false; /* lock failed or not found */
+
+ /* The entry is exclusively locked, so we can free the chunk first. */
+ dsa_free(area, ent->env);
+ dshash_delete_entry(pgStatSharedHash, ent);
+
+ return true;
+}
+
+
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
+ Assert(OidIsValid(databaseid));
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
+ for (p = envlist; *p != NULL; p++)
+ delete_stat_entry((*p)->type, (*p)->databaseid, (*p)->objectid, true);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ pfree(envlist);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1306,20 +1414,47 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ /* Lookup the entries of the current database in the stats hash. */
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
+ for (p = envlist; *p != NULL; p++)
+ {
+ PgStatEnvelope *env = *p;
+ PgStat_StatDBEntry *dbstat;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+
+ switch (env->type)
+ {
+ case PGSTAT_TYPE_TABLE:
+ init_tabentry(env);
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ init_funcentry(env);
+ break;
+
+ case PGSTAT_TYPE_DB:
+ init_dbentry(env);
+ dbstat = (PgStat_StatDBEntry *) &env->body;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ break;
+ default:
+ Assert(false);
+ }
+
+ LWLockRelease(&env->lock);
+ }
+
+ pfree(envlist);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1328,29 +1463,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ /* Reset the bgwriter statistics for the cluster. */
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1359,17 +1502,42 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ env = get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid,
+ false, NULL, NULL);
+ Assert(env);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&env->lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ env = get_stat_entry(stattype, MyDatabaseId, objoid, false, NULL, NULL);
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ if (env->type == PGSTAT_TYPE_TABLE)
+ init_tabentry(env);
+ else
+ {
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ init_funcentry(env);
+ }
+ LWLockRelease(&env->lock);
}
/* ----------
@@ -1383,48 +1551,63 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ ts = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Store the last autovacuum time in the database's hash table entry.
+ */
+ dbentry = get_local_dbstat_entry(dboid);
+ dbentry->last_autovac_time = ts;
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_TableStatus *tabentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+ tabentry = get_local_tabstat_entry(tableoid, shared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->t_counts.autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->t_counts.vacuum_count++;
+ }
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1435,9 +1618,10 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_TableStatus *tabentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1445,10 +1629,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1466,158 +1650,172 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ tabentry = get_local_tabstat_entry(RelationGetRelid(rel),
+ rel->rd_rel->relisshared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->t_counts.reset_changed_tuples = true;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.analyze_count++;
+ }
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-
-/* --------
- * pgstat_report_checksum_failures_in_db() -
- *
- * Tell the collector about one or more checksum failures.
- * --------
- */
-void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
-{
- PgStat_MsgChecksumFailure msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
-
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
-/* ----------
- * pgstat_ping() -
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Send some junk data to the collector to increase traffic.
- * ----------
+ * Reports about one or more checksum failures.
+ * --------
*/
void
-pgstat_ping(void)
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgDummy msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dbentry = get_local_dbstat_entry(dboid);
+
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
{
+ PgStatEnvelope *env;
PgStat_BackendFunctionEntry *htabent;
bool found;
@@ -1628,26 +1826,15 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
+ htabent = (PgStat_BackendFunctionEntry *) &env->body;
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
if (!found)
MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent->f_id = fcinfo->flinfo->fn_oid;
+
fcu->fs = &htabent->f_counts;
/* save stats for this function, later used to compensate for recursion */
@@ -1660,31 +1847,38 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
+ PgStatEnvelope *env;
+
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+ if (!env)
return NULL;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ return (PgStat_BackendFunctionEntry *) &env->body;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1725,9 +1919,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1739,8 +1930,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1760,7 +1950,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1776,116 +1967,157 @@ pgstat_initstats(Relation rel)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
-/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize body part of the returned envelope.
+ * ----------
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+static PgStatEnvelope *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
+ PgStatHashEntryKey key;
+ PgStatLocalHashEntry *entry;
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ if (pgStatLocalHash == NULL)
{
HASHCTL ctl;
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = sizeof(PgStatLocalHashEntry);
+
+ pgStatLocalHash = hash_create("Local stat entries",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ entry = hash_search(pgStatLocalHash, &key,
+ create ? HASH_ENTER : HASH_FIND, found);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ {
+ int len = pgstat_localentsize[type];
+
+ entry->env = MemoryContextAlloc(CacheMemoryContext,
+ PgStatEnvelopeSize(len));
+ entry->env->type = type;
+ entry->env->len = len;
}
+ return entry->env;
+}
+
+/* ----------
+ * get_local_dbstat_entry() -
+ *
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
+ */
+static PgStat_StatDBEntry *
+get_local_dbstat_entry(Oid dbid)
+{
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ bool found;
+
/*
* Find an entry or create a new one.
*/
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
+ env = get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
if (!found)
{
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
+ dbentry->databaseid = dbid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
}
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
+ return dbentry;
+}
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
+ */
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
+{
+ PgStatEnvelope *env;
+ PgStat_TableStatus *tabentry;
+ bool found;
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- return entry;
+ tabentry = (PgStat_TableStatus *) &env->body;
+
+ if (!found)
+ {
+ tabentry->t_id = rel_id;
+ tabentry->t_shared = isshared;
+ tabentry->trans = NULL;
+ MemSet(&tabentry->t_counts, 0, sizeof(PgStat_TableCounts));
+ tabentry->vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ }
+
+ return tabentry;
}
+
/*
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel from the current
+ * database then from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry, return NULL, don't create a new one
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStatEnvelope *env;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!env)
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
+ if (env)
+ return (PgStat_TableStatus *) &env->body;
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return NULL;
}
/*
@@ -2362,7 +2594,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2398,7 +2630,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2415,88 +2647,176 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
}
+/* ----------
+ * snapshot_statentry() -
+ *
+ * Common routine for functions pgstat_fetch_stat_*entry()
+ *
+ * Returns the pointer to the snapshot of the shared entry for the key or NULL
+ * if not found. Returned snapshots are stable during the current transaction
+ * or until pgstat_clear_snapshot() is called.
+ *
+ * Created snapshots are stored in pgStatSnapshotHash.
+ */
+static void *
+snapshot_statentry(const PgStatTypes type, const Oid dbid, const Oid objid)
+{
+ PgStatSnapshot *snap = NULL;
+ bool found;
+ PgStatHashEntryKey key;
+ size_t statentsize = pgstat_entsize[type];
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ /*
+ * Create new hash, with rather arbitrary initial number of entries since
+ * we don't know how this hash will grow.
+ */
+ if (!pgStatSnapshotHash)
+ {
+ HASHCTL ctl;
+
+ /*
+ * Create the hash in the stats context
+ *
+ * The entry is prepended by common header part represented by
+ * PgStatSnapshot.
+ */
+
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = PgStatSnapshotSize(statentsize);
+ ctl.hcxt = pgStatSnapshotContext;
+ pgStatSnapshotHash = hash_create("pgstat snapshot hash", 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ /* Find a snapshot */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ snap = hash_search(pgStatSnapshotHash, &key, HASH_ENTER, &found);
+
+ /*
+ * Refer shared hash if not found in the snapshot hash.
+ *
+ * In transaction state, it is obvious that we should create a snapshot
+ * entriy for consistency. If we are not, we return an up-to-date entry.
+ * Having said that, we need a snapshot since shared stats entry can be
+ * modified anytime. We share the same snapshot entry for the purpose.
+ */
+ if (!found || !IsTransactionState())
+ {
+ PgStatEnvelope *shenv;
+
+ shenv = get_stat_entry(type, dbid, objid, true, NULL, NULL);
+
+ if (shenv)
+ memcpy(&snap->body, &shenv->body, statentsize);
+
+ snap->negative = !shenv;
+ }
+
+ if (snap->negative)
+ return NULL;
+
+ return &snap->body;
+}
+
+
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *)
+ snapshot_statentry(PGSTAT_TYPE_DB, dbid, InvalidOid);
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_snapshot() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_snapshot(bool shared, Oid reloid)
+{
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(PGSTAT_TYPE_TABLE, dboid, reloid);
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2510,24 +2830,48 @@ pgstat_fetch_stat_tabentry(Oid relid)
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
-
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
-
- return funcentry;
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, func_id);
}
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ * ----------
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+
+ attach_shared_stats();
+
+ /* Nothing to do if already done */
+ if (global_snapshot_is_valid)
+ return;
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ memcpy(&snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ LWLockRelease(StatsLock);
+
+ global_snapshot_is_valid = true;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
+}
/* ----------
* pgstat_fetch_stat_beentry() -
@@ -2599,9 +2943,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return &snapshot_archiverStats;
}
@@ -2616,9 +2961,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return &snapshot_globalStats;
}
@@ -2832,8 +3178,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3009,12 +3355,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3027,7 +3376,7 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
@@ -3044,6 +3393,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3304,7 +3655,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3599,9 +3951,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
break;
@@ -4230,94 +4579,71 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Report archiver statistics
* ----------
*/
void
-pgstat_send_archiver(const char *xlog, bool failed)
+pgstat_report_archiver(const char *xlog, bool failed)
{
- PgStat_MsgArchiver msg;
+ TimestampTz now = GetCurrentTimestamp();
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ if (failed)
+ {
+ /* Failed archival attempt */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ /* Successful archival operation */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *l = &BgWriterStats;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += l->timed_checkpoints;
+ shared_globalStats->requested_checkpoints += l->requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += l->checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += l->checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += l->buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += l->buf_written_clean;
+ shared_globalStats->maxwritten_clean += l->maxwritten_clean;
+ shared_globalStats->buf_written_backend += l->buf_written_backend;
+ shared_globalStats->buf_fsync_backend += l->buf_fsync_backend;
+ shared_globalStats->buf_alloc += l->buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4326,425 +4652,30 @@ pgstat_send_bgwriter(void)
}
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
-{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
-
- return result;
-}
-
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
-{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->inserts_since_vacuum = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
-}
-
-
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
elog(DEBUG2, "writing stats file \"%s\"", statfile);
+ create_missing_dbentries();
+
/*
* Open the statistics temp file to write out the current values.
*/
@@ -4761,7 +4692,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4773,32 +4704,31 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_DB, InvalidOid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &(*penv)->body;
+
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_database_stats(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
@@ -4809,6 +4739,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
+
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4841,55 +4773,19 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
}
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
+ * pgstat_write_database_stats() -
+ * Write the stat file for a single database.
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_database_stats(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -4897,8 +4793,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -4925,24 +4821,31 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_TABLE, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatTabEntry *tabentry = (PgStat_StatTabEntry *) &(*penv)->body;
+
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_FUNCTION, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatFuncEntry *funcentry =
+ (PgStat_StatFuncEntry *) &(*penv)->body;
+
fputc('F', fpout);
rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* No more output to be done. Close the temp file and replace the old
@@ -4976,94 +4879,165 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
+}
- if (permanent)
+/* ----------
+ * create_missing_dbentries() -
+ *
+ * There may be the case where database entry is missing for the database
+ * where object stats are recorded. This function creates such missing
+ * dbentries so that so that all stats entries can be written out to files.
+ * ----------
+ */
+static void
+create_missing_dbentries(void)
+{
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ HTAB *oidhash;
+ HASHCTL ctl;
+ HASH_SEQ_STATUS scan;
+ Oid *poid;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = CurrentMemoryContext;
+ oidhash = hash_create("Temporary table of OIDs",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Collect OID from the shared stats hash */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ hash_search(oidhash, &p->key.databaseid, HASH_ENTER, NULL);
+ dshash_seq_term(&hstat);
+
+ /* Create missing database entries if not exists. */
+ hash_seq_init(&scan, oidhash);
+ while ((poid = (Oid *) hash_seq_search(&scan)) != NULL)
+ (void) get_stat_entry(PGSTAT_TYPE_DB, *poid, InvalidOid,
+ false, init_dbentry, NULL);
+
+ hash_destroy(oidhash);
+}
+
+
+/* ----------
+ * get_stat_entry() -
+ *
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new envelope. If found is not NULL, it is set to true if
+ * existing entry is found or false if not.
+ * ----------
+ */
+static PgStatEnvelope *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait, entry_initializer initfunc, bool *found)
+{
+ bool create = (initfunc != NULL);
+ PgStatHashEntry *shent;
+ PgStatEnvelope *shenv = NULL;
+ PgStatHashEntryKey key;
+ bool myfound;
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ shent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &myfound);
+ if (shent)
{
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
+ if (create && !myfound)
+ {
+ /* Create new stats envelope. */
+ size_t envsize = PgStatEnvelopeSize(pgstat_entsize[type]);
+ dsa_pointer chunk = dsa_allocate0(area, envsize);
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
+ shenv = dsa_get_address(area, chunk);
+ shenv->type = type;
+ shenv->databaseid = dbid;
+ shenv->objectid = objid;
+ shenv->len = pgstat_entsize[type];
+ LWLockInitialize(&shenv->lock, LWTRANCHE_STATS);
+
+ /*
+ * The lock on dshsh is released just after. Call initializer
+ * callback before it is exposed to other process.
+ */
+ if (initfunc)
+ initfunc(shenv);
+
+ /* Link the new entry from the hash entry. */
+ shent->env = chunk;
+ }
+ else
+ shenv = dsa_get_address(area, shent->env);
+
+ dshash_release_lock(pgStatSharedHash, shent);
}
+
+ if (found)
+ *found = myfound;
+
+ return shenv;
}
+
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * Reads in existing activity statistics files into the shared stats hash.
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
+ PgStatEnvelope *env;
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5072,7 +5046,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5080,38 +5054,30 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5125,7 +5091,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
fpin) != offsetof(PgStat_StatDBEntry, tables))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5134,76 +5100,33 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+
+ env = get_stat_entry(PGSTAT_TYPE_DB, dbbuf.databaseid,
+ InvalidOid,
+ false, init_dbentry, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ /* don't allow duplicate dbentries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, tables));
+ /* Read the data from the database-specific file. */
+ pgstat_read_db_statsfile(dbentry);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5213,59 +5136,50 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
/* ----------
* pgstat_read_db_statsfile() -
*
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry)
{
+ PgStatEnvelope *env;
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
FILE *fpin;
int32 format_id;
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5278,14 +5192,14 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5298,25 +5212,21 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE,
+ dbentry->databaseid, tabbuf.tableid,
+ false, init_tabentry, &found);
+ tabentry = (PgStat_StatTabEntry *) &env->body;
+ /* don't allow duplicate entries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5332,25 +5242,20 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE, dbentry->databaseid,
+ funcbuf.functionid,
+ false, init_funcentry, &found);
+ funcentry = (PgStat_StatFuncEntry *) &env->body;
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5366,7 +5271,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5374,292 +5279,15 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
-done:
FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
-}
-
-/* ----------
- * pgstat_setup_memcxt() -
- *
- * Create pgStatLocalContext, if not already done.
- * ----------
- */
-static void
-pgstat_setup_memcxt(void)
-{
- if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
@@ -5678,756 +5306,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum = tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- tabentry->inserts_since_vacuum = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum += tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * It is quite possible that a non-aggressive VACUUM ended up skipping
- * various pages, however, we'll zero the insert counter here regardless.
- * It's currently used only to track when we need to perform an
- * "insert" autovacuum, which are mainly intended to freeze newly inserted
- * tuples. Zeroing this may just mean we'll not try to vacuum the table
- * again until enough tuples have been inserted to trigger another insert
- * autovacuum. An anti-wraparound autovacuum will catch any persistent
- * stragglers.
- */
- tabentry->inserts_since_vacuum = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
}
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
+ if (pgStatSnapshotContext)
{
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
+ MemoryContextReset(pgStatSnapshotContext);
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
+ /* Reset variables that pointed to the context */
+ global_snapshot_is_valid = false;
+ pgStatSnapshotHash = NULL;
}
}
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
-}
-
/*
* Convert a potentially unsafely truncated activity string (see
* PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a4b9d212a2..83ca2c1113 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -255,7 +255,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -503,7 +502,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1326,12 +1324,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1780,11 +1772,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2694,8 +2681,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3058,8 +3043,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3126,13 +3109,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3205,22 +3181,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3681,22 +3641,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3892,8 +3836,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3928,8 +3870,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4130,8 +4071,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5109,18 +5048,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5239,12 +5166,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6139,7 +6060,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6195,8 +6115,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6431,7 +6349,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index e72d607a23..ea5030452d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1948,7 +1948,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ BgWriterStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2058,7 +2058,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2248,7 +2248,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2256,7 +2256,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 427b0d59cd..58a442f482 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 4c14e51c67..f61fd3e8ad 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -523,6 +523,7 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
+ LWLockRegisterTranche(LWTRANCHE_STATS, "activity_statistics");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index db47843229..97eccb35d3 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -49,3 +49,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
LogicalRepWorkerLock 43
CLogTruncationLock 44
+StatsLock 45
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5b677863b9..4461214ff3 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3192,6 +3192,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3766,6 +3772,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4204,6 +4211,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4216,8 +4225,13 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4252,7 +4266,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4260,6 +4274,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6d66ff8b44..b59b22c367 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -33,9 +33,6 @@
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1260,7 +1257,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1276,7 +1273,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1292,7 +1289,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1308,7 +1305,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1324,7 +1321,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1340,7 +1337,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1356,7 +1353,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1372,7 +1369,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1388,7 +1385,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1421,7 +1418,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1437,7 +1434,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1452,7 +1449,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1467,7 +1464,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1482,7 +1479,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1497,7 +1494,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1512,7 +1509,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1527,11 +1524,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1546,7 +1543,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1564,7 +1561,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1601,7 +1598,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1617,7 +1614,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..51748c99ad 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index f4247ea70d..f65d05c24c 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -631,6 +632,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1241,6 +1244,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..927ae319b1 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 106;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 619b2f9c71..9f1de1e42f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,8 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 763c1ee2bd..69bd794806 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -15,9 +15,11 @@
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,33 +43,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -78,9 +53,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -111,18 +85,17 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_delta_live_tuples;
PgStat_Counter t_delta_dead_tuples;
PgStat_Counter t_changed_tuples;
+ bool reset_changed_tuples;
PgStat_Counter t_blocks_fetched;
PgStat_Counter t_blocks_hit;
+
+ PgStat_Counter vacuum_count;
+ PgStat_Counter autovac_vacuum_count;
+ PgStat_Counter analyze_count;
+ PgStat_Counter autovac_analyze_count;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -156,6 +129,10 @@ typedef struct PgStat_TableStatus
Oid t_id; /* table's OID */
bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
} PgStat_TableStatus;
@@ -181,280 +158,32 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgTempFile
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter buf_alloc;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -486,96 +215,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -584,13 +225,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9D
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -600,7 +237,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -610,29 +246,55 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ Oid databaseid;
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/* ----------
+ * PgStat_HashMountInfo dshash mount information
+ * ----------
+ */
+typedef struct PgStat_HashMountInfo
+{
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
+} PgStat_HashMountInfo;
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
Oid tableid;
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -652,19 +314,15 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -679,7 +337,7 @@ typedef struct PgStat_StatFuncEntry
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -695,7 +353,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -761,7 +419,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1005,7 +662,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1202,13 +859,15 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1223,29 +882,26 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
@@ -1406,8 +1062,8 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1416,11 +1072,15 @@ extern void pgstat_send_bgwriter(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_snapshot(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 8fda8e4f78..13371e8cb7 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -220,6 +220,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_SXACT,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 83a15f6795..77d1572a99 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.18.2
* [PATCH v30 5/7] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 500ms and such large data is
serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 500ms. Locks
on the shared statistics is acquired by the units of such like tables,
functions so the expected chance of collision are not so
high. Furthermore, until 1 second has elapsed since the last flushing
to shared stats, lock failure postpones stats flushing so that lock
contention doesn't slow down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/heap/heapam_handler.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/access/transam/xlog.c | 4 +-
src/backend/catalog/index.c | 24 +-
src/backend/commands/analyze.c | 8 +-
src/backend/commands/dbcommands.c | 2 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/vacuum.c | 4 +-
src/backend/postmaster/autovacuum.c | 74 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 46 +-
src/backend/postmaster/pgarch.c | 14 +-
src/backend/postmaster/pgstat.c | 4869 +++++++----------
src/backend/postmaster/postmaster.c | 88 +-
src/backend/replication/basebackup.c | 4 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 1 +
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/smgr.c | 12 +-
src/backend/tcop/postgres.c | 37 +-
src/backend/utils/adt/pgstatfuncs.c | 55 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 3 -
src/backend/utils/init/postinit.c | 11 +
src/backend/utils/misc/guc.c | 12 +-
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 3 +-
src/include/pgstat.h | 514 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/guc_tables.h | 2 +-
src/include/utils/timeout.h | 1 +
33 files changed, 2136 insertions(+), 3689 deletions(-)
inline patch 1: src/backend/access/heap/heapam_handler.c, src/backend/access/heap/vacuumlazy.c, src/backend/access/transam/xlog.c and 30 more (+2136/-3689 lines) download inline-1.patch
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ca52846b97..d1be516db4 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1074,8 +1074,8 @@ heapam_scan_analyze_next_tuple(TableScanDesc scan, TransactionId OldestXmin,
* our own. In this case we should count and sample the row,
* to accommodate users who load a table and analyze it in one
* transaction. (pgstat_report_analyze has to adjust the
- * numbers we send to the stats collector to make this come
- * out right.)
+ * numbers we report to the activity stats facility to make
+ * this come out right.)
*/
if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(targtuple->t_data)))
{
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 04b12342b8..36e2d542f6 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -584,7 +584,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
new_min_multi,
false);
- /* report results to the stats collector, too */
+ /* report results to the activity stats facility, too */
pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
new_live_tuples,
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index d75a257da2..99beaf0215 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8594,9 +8594,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bd7ec923e9..46fe9fd85f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1699,28 +1699,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be flushed by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Close relations */
table_close(pg_class, RowExclusiveLock);
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 924ef37c81..06b03cb8e1 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -655,10 +655,10 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
}
/*
- * Report ANALYZE to the stats collector, too. However, if doing
- * inherited stats we shouldn't report, because the stats collector only
- * tracks per-table stats. Reset the changes_since_analyze counter only
- * if we analyzed all columns; otherwise, there is still work for
+ * Report ANALYZE to the activity stats facility, too. However, if doing
+ * inherited stats we shouldn't report, because the activity stats facility
+ * only tracks per-table stats. Reset the changes_since_analyze counter
+ * only if we analyzed all columns; otherwise, there is still work for
* auto-analyze to do.
*/
if (!inh)
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 367c30adb0..a18e7068ae 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -971,7 +971,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
DropDatabaseBuffers(db_id);
/*
- * Tell the stats collector to forget it immediately, too.
+ * Tell the active stats facility to forget it immediately, too.
*/
pgstat_drop_database(db_id);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index e5a5eef102..4151fa5335 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -336,10 +336,10 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
/*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
+ * Inform activity stats facility about our activity: basically, we
+ * truncated the matview and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
*/
pgstat_count_truncate(matviewRel);
if (!stmt->skipData)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 59731d687f..4e4d34d63b 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -321,8 +321,8 @@ vacuum(List *relations, VacuumParams *params,
errmsg("VACUUM option DISABLE_PAGE_SKIPPING cannot be used with FULL")));
/*
- * Send info about dead objects to the statistics collector, unless we are
- * in autovacuum --- autovacuum.c does this for itself.
+ * Send info about dead objects to the activity statistics facility, unless
+ * we are in autovacuum --- autovacuum.c does this for itself.
*/
if ((params->options & VACOPT_VACUUM) && !IsAutoVacuumWorkerProcess())
pgstat_vacuum_stat();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7e97ffab27..19a2357f0d 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -338,9 +338,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1665,12 +1662,12 @@ AutoVacWorkerMain(int argc, char *argv[])
char dbname[NAMEDATALEN];
/*
- * Report autovac startup to the stats collector. We deliberately do
- * this before InitPostgres, so that the last_autovac_time will get
- * updated even if the connection attempt fails. This is to prevent
- * autovac from getting "stuck" repeatedly selecting an unopenable
- * database, rather than making any progress on stuff it can connect
- * to.
+ * Report autovac startup to the activity stats facility. We
+ * deliberately do this before InitPostgres, so that the
+ * last_autovac_time will get updated even if the connection attempt
+ * fails. This is to prevent autovac from getting "stuck" repeatedly
+ * selecting an unopenable database, rather than making any progress on
+ * stuff it can connect to.
*/
pgstat_report_autovac(dbid);
@@ -1938,8 +1935,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1958,17 +1953,11 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
/*
- * Clean up any dead statistics collector entries for this DB. We always
+ * Clean up any dead activity statistics entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
* nothing worth vacuuming in the database.
*/
@@ -2011,9 +2000,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2092,8 +2078,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2176,8 +2162,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2736,29 +2722,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2779,17 +2742,12 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
bool doanalyze;
autovac_table *tab = NULL;
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
bool wraparound;
AutoVacOpts *avopts;
/* use fresh stats */
autovac_refresh_stats();
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* fetch the relation's relcache entry */
classTup = SearchSysCacheCopy1(RELOID, ObjectIdGetDatum(relid));
if (!HeapTupleIsValid(classTup))
@@ -2813,8 +2771,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
}
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2936,7 +2894,7 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
*
* For analyze, the analysis done is that the number of tuples inserted,
* deleted and updated since the last analyze exceeds a threshold calculated
- * in the same fashion as above. Note that the collector actually stores
+ * in the same fashion as above. Note that the activity statistics stores
* the number of tuples (both live and dead) that there were as of the last
* analyze. This is asymmetric to the VACUUM case.
*
@@ -2946,8 +2904,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
*
* A table whose autovacuum_enabled option is false is
* automatically skipped (unless we have to vacuum it due to freeze_max_age).
- * Thus autovacuum can be disabled for specific tables. Also, when the stats
- * collector does not have data about a table, it will be skipped.
+ * Thus autovacuum can be disabled for specific tables. Also, when the activity
+ * statistics does not have data about a table, it will be skipped.
*
* A table whose vac_base_thresh value is < 0 takes the base value from the
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..4382b1726f 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -234,9 +234,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Send off activity statistics to the activity stats facility
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..12f06a316d 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -350,7 +350,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ BgWriterStats.requested_checkpoints++;
}
/*
@@ -364,7 +364,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ BgWriterStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -486,13 +486,13 @@ CheckpointerMain(void)
CheckArchiveTimeout();
/*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
+ * Send off activity statistics to the activity stats facility. (The
+ * reason why we re-use bgwriter-related code for this is that the
+ * bgwriter and checkpointer used to be just one process. It's
+ * probably not worth the trouble to split the stats support into two
+ * independent stats message types.)
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* Sleep until we are signaled or it's time for another checkpoint or
@@ -533,29 +533,29 @@ HandleCheckpointerInterrupts(void)
ProcessConfigFile(PGC_SIGHUP);
/*
- * Checkpointer is the last process to shut down, so we ask it to
- * hold the keys for a range of other tasks required most of which
- * have nothing to do with checkpointing at all.
+ * Checkpointer is the last process to shut down, so we ask it to hold
+ * the keys for a range of other tasks required most of which have
+ * nothing to do with checkpointing at all.
*
- * For various reasons, some config values can change dynamically
- * so the primary copy of them is held in shared memory to make
- * sure all backends see the same value. We make Checkpointer
- * responsible for updating the shared memory copy if the
- * parameter setting changes because of SIGHUP.
+ * For various reasons, some config values can change dynamically so
+ * the primary copy of them is held in shared memory to make sure all
+ * backends see the same value. We make Checkpointer responsible for
+ * updating the shared memory copy if the parameter setting changes
+ * because of SIGHUP.
*/
UpdateSharedMemoryConfig();
}
if (ShutdownRequestPending)
{
/*
- * From here on, elog(ERROR) should end with exit(1), not send
- * control back to the sigsetjmp block above
+ * From here on, elog(ERROR) should end with exit(1), not send control
+ * back to the sigsetjmp block above
*/
ExitOnAnyError = true;
/* Close down the database */
ShutdownXLOG(0, 0);
/* Normal exit from the checkpointer is here */
- proc_exit(0); /* done */
+ proc_exit(0); /* done */
}
}
@@ -691,9 +691,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Report interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1238,8 +1238,8 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ BgWriterStats.buf_written_backend += CheckpointerShmem->num_backend_writes;
+ BgWriterStats.buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 7b66cb5613..f271714f45 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -173,7 +173,7 @@ PgArchiverMain(void)
XLogArchiveWakeupStart();
on_shmem_exit(PgArchiverKill, 0);
-
+
pgarch_MainLoop();
exit(0);
@@ -393,20 +393,20 @@ pgarch_ArchiverCopyLoop(void)
pgarch_archiveDone(xlog);
/*
- * Tell the collector about the WAL file that we successfully
- * archived
+ * Tell the activity statistics facility about the WAL file
+ * that we successfully archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
else
{
/*
- * Tell the collector about the WAL file that we failed to
- * archive
+ * Tell the activity statistics facility about the WAL file
+ * that we failed to archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index ab42df7e1b..6e76cc40bc 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (1000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (250ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (10000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,68 +35,43 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 1000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 250 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 10000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -116,7 +86,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -131,75 +100,163 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-
-/* ----------
- * Local data
- * ----------
+ * Shared stats bootstrap information, protected by StatsLock.
+ *
+ * refcount is used to know whether a process going to detach shared stats is
+ * the last process or not. The last process writes out the stats files.
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ dsa_pointer global_stats; /* DSA pointer to global stats */
+ dsa_pointer archiver_stats; /* Ditto for archiver stats */
+ int refcount; /* # of processes that is attaching the shared
+ * stats memory */
+} StatsShmemStruct;
-static time_t last_pgstat_start_time;
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
+ * Enums and types to define shared statistics structure.
+ *
+ * Statistics entries for each object is stored in individual DSA-allocated
+ * memory. Every entry is pointed from the dshash pgStatSharedHash via
+ * dsa_pointer. The structure makes object-stats entries not moved by dshash
+ * resizing, and allows the dshash can release lock sooner on stats
+ * updates. Also it reduces interfering among write-locks on each stat entry by
+ * not relying on partition lock of dshash. PgStatLocalHashEntry is the
+ * local-stats equivalent of PgStatHashEntry for shared stat entries.
+ *
+ * Each stat entry is enveloped by the type PgStatEnvelope, which stores common
+ * attribute of all kind of statistics and a LWLock lock object.
*
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStatEnvelope (dsa memory block)
+ *
+ * Local stats are stored as:
+ *
+ * dshash pgStatLocalHash
+ * -> PgStatLocalHashEntry (dynahash entry)
+ * (direct pointer)-> PgStatEnvelope (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries. */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_ALL, /* Not a type, for the parameters of
+ * pgstat_collect_stat_entries */
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION, /* per-function statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static size_t pgstat_entsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry) /* PGSTAT_TYPE_FUNCTION */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static size_t pgstat_localentsize[] =
+{
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry) /* PGSTAT_TYPE_FUNCTION */
+};
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashEntryKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object OID */
+} PgStatHashEntryKey;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Stats numbers that are waiting for flushing out to shared stats are held in
+ * pgStatLocalHash,
*/
-static HTAB *pgStatFunctions = NULL;
+typedef struct PgStatHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ dsa_pointer env; /* pointer to shared stats envelope in DSA */
+} PgStatHashEntry;
+
+/* struct for shared statistics entry pointed from shared hash entry. */
+typedef struct PgStatEnvelope
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* databaseid */
+ Oid objectid; /* objectid */
+ size_t len; /* length of body, fixed per type. */
+ LWLock lock; /* lightweight lock to protect body */
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatEnvelope;
+
+#define PgStatEnvelopeSize(bodylen) \
+ (offsetof(PgStatEnvelope, body) + (bodylen))
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ PgStatEnvelope *env; /* pointer to stats envelope in heap */
+} PgStatLocalHashEntry;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * Snapshot is stats entry that is locally copied to offset stable values for a
+ * transaction.
*/
-static bool have_function_stats = false;
+typedef struct PgStatSnapshot
+{
+ PgStatHashEntryKey key;
+ bool negative;
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatSnapshot;
+
+#define PgStatSnapshotSize(bodylen) \
+ (offsetof(PgStatSnapshot, body) + (bodylen))
+
+/* parameter for shared hashes */
+static const dshash_parameters dsh_rootparams = {
+ sizeof(PgStatHashEntryKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
+
+/* Local stats numbers are stored here */
+static HTAB *pgStatLocalHash = NULL;
+
+#define HAVE_ANY_PENDING_STATS() \
+ (pgStatLocalHash != NULL || \
+ pgStatXactCommit != 0 || pgStatXactRollback != 0)
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -236,11 +293,10 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatSnapshotHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -249,19 +305,17 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are collected not per database nor per table
+ * basis. shared_* points to shared memory and snapshot_* are backend
+ * snapshots.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+static bool global_snapshot_is_valid = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats snapshot_globalStats;
/*
* Total time charged to functions so far in the current backend.
@@ -270,523 +324,269 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/*
+ * Newly created shared stats entries needs to be initialized before the other
+ * processes get access it. get_stat_entry() calls it for the purpose.
+ */
+typedef void (*entry_initializer) (PgStatEnvelope * env);
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStatEnvelope * get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait,
+ entry_initializer initfunc, bool *found);
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid);
+static void create_missing_dbentries(void);
+static void pgstat_write_database_stats(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry);
+
+static bool flush_tabstat(PgStatEnvelope * env, bool nowait);
+static bool flush_funcstat(PgStatEnvelope * env, bool nowait);
+static bool flush_dbstat(PgStatEnvelope * env, bool nowait);
+
+static void init_dbentry(PgStatEnvelope * env);
+static void init_funcentry(PgStatEnvelope * env);
+static void init_tabentry(PgStatEnvelope * env);
+
+static bool delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait);
+static PgStatEnvelope * get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static void pgstat_snapshot_global_stats(void);
+
static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
-
-static void pgstat_setup_memcxt(void);
-
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
+ bool found;
-#define TESTBYTEVAL ((char) 199)
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
+}
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgStatLocalContext and pgStatSnapshotContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgStatLocalContext)
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+
+ if (!pgStatSnapshotContext)
+ pgStatSnapshotContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Database statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+}
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read saved statistics files if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * Don't use dsm under postmaster, or when not tracking counts.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ pgstat_setup_memcxt();
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ if (area)
+ return;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
+ /*
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that processes going to exit can find whether it
+ * is the last or not.
+ */
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
+ {
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_rootparams, 0);
- test_byte++; /* just make sure variable is changed */
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Load saved data if any. */
+ pgstat_read_statsfiles();
- /* If we get here, we have a working socket */
- break;
+ StatsShmem->refcount = 1;
}
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockRelease(StatsLock);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * If we're not the first process, attach existing shared stats area
+ * outside the StatsLock section.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (!area)
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* Attach shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_rootparams,
+ StatsShmem->hash_handle, 0);
+
+ /* Setup local variables */
+ pgStatLocalHash = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
}
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
- }
-
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+ global_snapshot_is_valid = false;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_stats)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
- {
- int nchars;
- Oid tmp_oid;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount < 1)
+ {
/*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
+ * The process is the last one that is attaching the shared stats
+ * memory. Write out the stats files if requested.
*/
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
+ if (write_stats)
+ pgstat_write_statsfiles();
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ /* No one is using the area. */
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
- FreeDir(dir);
+
+ LWLockRelease(StatsLock);
+
+ /*
+ * Detach the area. Automatically destroyed when the last process detached
+ * it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatLocalHash = NULL;
+ global_snapshot_is_valid = false;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
-#ifdef EXEC_BACKEND
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
-
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
/*
- * Okay, fork off the collector.
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
+ detach_shared_stats(false); /* Don't write files. */
+ attach_shared_stats();
}
/* ------------------------------------------------------------
@@ -794,144 +594,488 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait = !force; /* Don't use force ever after */
+ HASH_SEQ_STATUS scan;
+ PgStatLocalHashEntry *lent;
+ PgStatLocalHashEntry **dbentlist;
+ int dbentlistlen = 8;
+ int ndbentries = 0;
+ int remains = 0;
int i;
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (area == NULL || !HAVE_ANY_PENDING_STATS())
+ return 0;
+
+ dbentlist = palloc(sizeof(PgStatLocalHashEntry *) * dbentlistlen);
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
+ if (nowait)
+ {
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
+ {
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ nowait = false;
+ }
/*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * flush_tabstat applies some of stats numbers of flushed entries into
+ * local database stats. So flush-out database stats later.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ if (pgStatLocalHash)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /* Step 1: flush out other than database stats */
+ hash_seq_init(&scan, pgStatLocalHash);
+ while ((lent = (PgStatLocalHashEntry *) hash_seq_search(&scan)) != NULL)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
+ bool remove = false;
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
+ switch (lent->env->type)
+ {
+ case PGSTAT_TYPE_DB:
+ if (ndbentries >= dbentlistlen)
+ {
+ dbentlistlen *= 2;
+ dbentlist = repalloc(dbentlist,
+ sizeof(PgStatLocalHashEntry *) *
+ dbentlistlen);
+ }
+ dbentlist[ndbentries++] = lent;
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent->env, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent->env, nowait))
+ remove = true;
+ break;
+ default:
+ Assert(false);
+ }
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
+ }
+
+ /* Step 2: flush out database stats */
+ for (i = 0; i < ndbentries; i++)
+ {
+ PgStatLocalHashEntry *lent = dbentlist[i];
+
+ if (flush_dbstat(lent->env, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ pfree(dbentlist);
+
+ if (remains <= 0)
+ {
+ hash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
+ }
+
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (shared_globalStats->stats_timestamp < now)
+ shared_globalStats->stats_timestamp = now;
+ LWLockRelease(StatsLock);
+
+ /*
+ * If we have pending local stats, let the caller know the retry interval.
+ */
+ if (HAVE_ANY_PENDING_STATS())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
+
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
+
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
+
+ return retry_interval / 1000;
}
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
+}
+
+/*
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_tabstat(PgStatEnvelope * lenv, bool nowait)
+{
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+ bool found;
+
+ Assert(lenv->type == PGSTAT_TYPE_TABLE);
+
+ lstats = (PgStat_TableStatus *) &lenv->body;
+ dboid = lstats->t_shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_TABLE, dboid, lstats->t_id,
+ nowait, init_tabentry, &found);
+
+ /* skip if dshash failed to acquire lock */
+ if (shenv == NULL)
+ return false;
+
+ /* retrieve the shared table stats entry from the envelope */
+ shtabstats = (PgStat_StatTabEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated ||
+ lstats->t_counts.vacuum_count > 0 ||
+ lstats->t_counts.analyze_count > 0 ||
+ lstats->t_counts.autovac_vacuum_count > 0 ||
+ lstats->t_counts.autovac_analyze_count > 0)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ /* clear the change counter if requested */
+ if (lstats->t_counts.reset_changed_tuples)
+ shtabstats->changes_since_analyze = 0;
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /*
+ * Update vacuum/analyze timestamp and counters, so that the values won't
+ * goes back.
+ */
+ if (shtabstats->vacuum_timestamp < lstats->vacuum_timestamp)
+ shtabstats->vacuum_timestamp = lstats->vacuum_timestamp;
+ shtabstats->vacuum_count += lstats->t_counts.vacuum_count;
+
+ if (shtabstats->autovac_vacuum_timestamp < lstats->autovac_vacuum_timestamp)
+ shtabstats->autovac_vacuum_timestamp = lstats->autovac_vacuum_timestamp;
+ shtabstats->autovac_vacuum_count += lstats->t_counts.autovac_vacuum_count;
+
+ if (shtabstats->analyze_timestamp < lstats->analyze_timestamp)
+ shtabstats->analyze_timestamp = lstats->analyze_timestamp;
+ shtabstats->analyze_count += lstats->t_counts.analyze_count;
+
+ if (shtabstats->autovac_analyze_timestamp < lstats->autovac_analyze_timestamp)
+ shtabstats->autovac_analyze_timestamp = lstats->autovac_analyze_timestamp;
+ shtabstats->autovac_analyze_count += lstats->t_counts.autovac_analyze_count;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shenv->lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ return true;
+}
+
+/* ----------
+ * init_tabentry() -
+ *
+ * initializes table stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_tabentry(PgStatEnvelope * env)
+{
+ PgStat_StatTabEntry *tabent = (PgStat_StatTabEntry *) &env->body;
+
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * If it's a new table entry, initialize counters to the values we just
+ * got.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
-
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ Assert(env->type == PGSTAT_TYPE_TABLE);
+ tabent->tableid = env->objectid;
+ tabent->numscans = 0;
+ tabent->tuples_returned = 0;
+ tabent->tuples_fetched = 0;
+ tabent->tuples_inserted = 0;
+ tabent->tuples_updated = 0;
+ tabent->tuples_deleted = 0;
+ tabent->tuples_hot_updated = 0;
+ tabent->n_live_tuples = 0;
+ tabent->n_dead_tuples = 0;
+ tabent->changes_since_analyze = 0;
+ tabent->blocks_fetched = 0;
+ tabent->blocks_hit = 0;
+
+ tabent->vacuum_timestamp = 0;
+ tabent->vacuum_count = 0;
+ tabent->autovac_vacuum_timestamp = 0;
+ tabent->autovac_vacuum_count = 0;
+ tabent->analyze_timestamp = 0;
+ tabent->analyze_count = 0;
+ tabent->autovac_analyze_timestamp = 0;
+ tabent->autovac_analyze_count = 0;
}
+
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * flush_funcstat - flush out a local function stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatEnvelope * env, bool nowait)
+{
+ /* we assume this inits to all zeroes: */
+ static const PgStat_FunctionCounts all_zeroes;
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatFuncEntry *sharedent = NULL; /* shared stats entry */
+ bool found;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) &env->body;
+
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&localent->f_counts, &all_zeroes,
+ sizeof(PgStat_FunctionCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, localent->f_id,
+ nowait, init_funcentry, &found);
+ /* skip if dshash failed to acquire lock */
+ if (sharedent == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* retrieve the shared table stats entry from the envelope */
+ sharedent = (PgStat_StatFuncEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->f_numcalls += localent->f_counts.f_numcalls;
+ sharedent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ sharedent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shenv->lock);
+
+ return true;
+}
+
+
+/* ----------
+ * init_funcentry() -
+ *
+ * initializes function stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
*/
static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+init_funcentry(PgStatEnvelope * env)
{
- int n;
- int len;
+ PgStat_StatFuncEntry *shstat = (PgStat_StatFuncEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ shstat->functionid = env->objectid;
+ shstat->f_numcalls = 0;
+ shstat->f_total_time = 0;
+ shstat->f_self_time = 0;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatEnvelope * env, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStatEnvelope *shenv;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &env->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_DB, localent->databaseid, InvalidOid,
+ nowait, init_dbentry, NULL);
+
+ /* skip if dshash failed to acquire lock */
+ if (!shenv)
+ return false;
+
+ /* retrieve the shared stats entry from the envelope */
+ sharedent = (PgStat_StatDBEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(localent->databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -939,257 +1083,102 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&shenv->lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
+
+/* ----------
+ * init_dbentry() -
+ *
+ * initializes database stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_dbentry(PgStatEnvelope * env)
+{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+ dbentry->databaseid = env->databaseid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ /* initialize the new shared entry */
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
+}
+
+
/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+ * Create the filename for a DB stat file; filename is output parameter points
+ * to a character buffer of length len.
*/
static void
-pgstat_send_funcstats(void)
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
- {
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
-
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
}
/* ----------
- * pgstat_vacuum_stat() -
+ * collect_stat_entries() -
*
- * Will tell the collector about objects he can get rid of.
+ * Collect the shared statistics entries specified by type and dbid. Returns a
+ * list of pointer to shared statistics in palloc'ed memory. If type is
+ * PGSTAT_TYPE_ALL, all types of statistics of the database is collected. If
+ * type is PGSTAT_TYPE_DB, the parameter dbid is ignored and collect all
+ * PGSTAT_TYPE_DB entries.
* ----------
*/
-void
-pgstat_vacuum_stat(void)
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
- return;
-
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
-
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ int listlen = 16;
+ PgStatEnvelope **envlist = palloc(sizeof(PgStatEnvelope * *) * listlen);
+ int n = 0;
+
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ if ((type != PGSTAT_TYPE_ALL && p->key.type != type) ||
+ (type != PGSTAT_TYPE_DB && p->key.databaseid != dbid))
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
+ if (n >= listlen - 1)
{
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
+ listlen *= 2;
+ envlist = repalloc(envlist, listlen * sizeof(PgStatEnvelope * *));
}
+ envlist[n++] = dsa_get_address(area, p->env);
}
+ dshash_seq_term(&hstat);
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
+ envlist[n] = NULL;
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
- }
+ return envlist;
}
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
* into a temporary hash table. Caller should hash_destroy the result
@@ -1198,7 +1187,7 @@ pgstat_vacuum_stat(void)
* ----------
*/
static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
HTAB *htab;
HASHCTL hash_ctl;
@@ -1212,7 +1201,7 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
hash_ctl.entrysize = sizeof(Oid);
hash_ctl.hcxt = CurrentMemoryContext;
htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
+ PGSTAT_TABLE_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
@@ -1239,65 +1228,184 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
}
+/* ----------
+ * pgstat_vacuum_stat() -
+ *
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
+ * ----------
+ */
+void
+pgstat_vacuum_stat(void)
+{
+ HTAB *dbids; /* database ids */
+ HTAB *relids; /* relation ids in the current database */
+ HTAB *funcids; /* function ids in the current database */
+ PgStatEnvelope **victims; /* victim entry list */
+ int arraylen = 0; /* storage size of the above */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+ int i;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+
+ /* collect victims from shared stats */
+ arraylen = 16;
+ victims = palloc(sizeof(PgStatEnvelope * *) * arraylen);
+ nvictims = 0;
+
+ dshash_seq_init(&dshstat, pgStatSharedHash, false);
+
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ HTAB *oidtab;
+ Oid *key;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
+ continue;
+
+ switch (ent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidtab = dbids;
+ key = &ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidtab = relids;
+ key = &ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidtab = funcids;
+ key = &ent->key.objectid;
+ break;
+ case PGSTAT_TYPE_ALL:
+ Assert(false);
+ break;
+ }
+
+ /* Skip existent objects. */
+ if (hash_search(oidtab, key, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* extend the list if needed */
+ if (nvictims >= arraylen)
+ {
+ arraylen *= 2;
+ victims = repalloc(victims, sizeof(PgStatEnvelope * *) * arraylen);
+ }
+
+ victims[nvictims++] = dsa_get_address(area, ent->env);
+ }
+ dshash_seq_term(&dshstat);
+ hash_destroy(dbids);
+ hash_destroy(relids);
+ hash_destroy(funcids);
+
+ /* Now try removing the victim entries */
+ for (i = 0; i < nvictims; i++)
+ {
+ PgStatEnvelope *p = victims[i];
+
+ delete_stat_entry(p->type, p->databaseid, p->objectid, true);
+ }
+}
+
+
+/* ----------
+ * delete_stat_entry() -
+ *
+ * Deletes the specified entry from shared stats hash.
+ *
+ * Returns true when successfully deleted.
+ * ----------
+ */
+static bool
+delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait)
+{
+ PgStatHashEntryKey key;
+ PgStatHashEntry *ent;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ ent = dshash_find_extended(pgStatSharedHash, &key,
+ true, nowait, false, NULL);
+
+ if (!ent)
+ return false; /* lock failed or not found */
+
+ /* The entry is exclusively locked, so we can free the chunk first. */
+ dsa_free(area, ent->env);
+ dshash_delete_entry(pgStatSharedHash, ent);
+
+ return true;
+}
+
+
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
+ Assert(OidIsValid(databaseid));
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
+ for (p = envlist; *p != NULL; p++)
+ delete_stat_entry((*p)->type, (*p)->databaseid, (*p)->objectid, true);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ pfree(envlist);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1306,20 +1414,47 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ /* Lookup the entries of the current database in the stats hash. */
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
+ for (p = envlist; *p != NULL; p++)
+ {
+ PgStatEnvelope *env = *p;
+ PgStat_StatDBEntry *dbstat;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+
+ switch (env->type)
+ {
+ case PGSTAT_TYPE_TABLE:
+ init_tabentry(env);
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ init_funcentry(env);
+ break;
+
+ case PGSTAT_TYPE_DB:
+ init_dbentry(env);
+ dbstat = (PgStat_StatDBEntry *) &env->body;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ break;
+ default:
+ Assert(false);
+ }
+
+ LWLockRelease(&env->lock);
+ }
+
+ pfree(envlist);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1328,29 +1463,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ /* Reset the bgwriter statistics for the cluster. */
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1359,17 +1502,42 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ env = get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid,
+ false, NULL, NULL);
+ Assert(env);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&env->lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ env = get_stat_entry(stattype, MyDatabaseId, objoid, false, NULL, NULL);
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ if (env->type == PGSTAT_TYPE_TABLE)
+ init_tabentry(env);
+ else
+ {
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ init_funcentry(env);
+ }
+ LWLockRelease(&env->lock);
}
/* ----------
@@ -1383,48 +1551,63 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ ts = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Store the last autovacuum time in the database's hash table entry.
+ */
+ dbentry = get_local_dbstat_entry(dboid);
+ dbentry->last_autovac_time = ts;
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_TableStatus *tabentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+ tabentry = get_local_tabstat_entry(tableoid, shared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->t_counts.autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->t_counts.vacuum_count++;
+ }
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1435,9 +1618,10 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_TableStatus *tabentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1445,10 +1629,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1466,158 +1650,172 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ tabentry = get_local_tabstat_entry(RelationGetRelid(rel),
+ rel->rd_rel->relisshared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->t_counts.reset_changed_tuples = true;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.analyze_count++;
+ }
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-
-/* --------
- * pgstat_report_checksum_failures_in_db() -
- *
- * Tell the collector about one or more checksum failures.
- * --------
- */
-void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
-{
- PgStat_MsgChecksumFailure msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
-
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
-/* ----------
- * pgstat_ping() -
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Send some junk data to the collector to increase traffic.
- * ----------
+ * Reports about one or more checksum failures.
+ * --------
*/
void
-pgstat_ping(void)
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgDummy msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dbentry = get_local_dbstat_entry(dboid);
+
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
{
+ PgStatEnvelope *env;
PgStat_BackendFunctionEntry *htabent;
bool found;
@@ -1628,26 +1826,15 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
+ htabent = (PgStat_BackendFunctionEntry *) &env->body;
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
if (!found)
MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent->f_id = fcinfo->flinfo->fn_oid;
+
fcu->fs = &htabent->f_counts;
/* save stats for this function, later used to compensate for recursion */
@@ -1660,31 +1847,38 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
+ PgStatEnvelope *env;
+
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+ if (!env)
return NULL;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ return (PgStat_BackendFunctionEntry *) &env->body;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1725,9 +1919,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1739,8 +1930,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1760,7 +1950,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1776,116 +1967,157 @@ pgstat_initstats(Relation rel)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
-/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize body part of the returned envelope.
+ * ----------
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+static PgStatEnvelope *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
+ PgStatHashEntryKey key;
+ PgStatLocalHashEntry *entry;
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ if (pgStatLocalHash == NULL)
{
HASHCTL ctl;
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = sizeof(PgStatLocalHashEntry);
+
+ pgStatLocalHash = hash_create("Local stat entries",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ entry = hash_search(pgStatLocalHash, &key,
+ create ? HASH_ENTER : HASH_FIND, found);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ {
+ int len = pgstat_localentsize[type];
+
+ entry->env = MemoryContextAlloc(CacheMemoryContext,
+ PgStatEnvelopeSize(len));
+ entry->env->type = type;
+ entry->env->len = len;
}
+ return entry->env;
+}
+
+/* ----------
+ * get_local_dbstat_entry() -
+ *
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
+ */
+static PgStat_StatDBEntry *
+get_local_dbstat_entry(Oid dbid)
+{
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ bool found;
+
/*
* Find an entry or create a new one.
*/
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
+ env = get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
if (!found)
{
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
+ dbentry->databaseid = dbid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
}
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
+ return dbentry;
+}
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
+ */
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
+{
+ PgStatEnvelope *env;
+ PgStat_TableStatus *tabentry;
+ bool found;
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- return entry;
+ tabentry = (PgStat_TableStatus *) &env->body;
+
+ if (!found)
+ {
+ tabentry->t_id = rel_id;
+ tabentry->t_shared = isshared;
+ tabentry->trans = NULL;
+ MemSet(&tabentry->t_counts, 0, sizeof(PgStat_TableCounts));
+ tabentry->vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ }
+
+ return tabentry;
}
+
/*
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel from the current
+ * database then from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry, return NULL, don't create a new one
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStatEnvelope *env;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!env)
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
+ if (env)
+ return (PgStat_TableStatus *) &env->body;
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return NULL;
}
/*
@@ -2316,8 +2548,8 @@ AtPrepare_PgStat(void)
*
* All we need do here is unlink the transaction stats state from the
* nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
+ * reported to the activity stats facility immediately, while the effects on
+ * live and dead tuple counts are preserved in the 2PC state file.
*
* Note: AtEOXact_PgStat is not called during PREPARE.
*/
@@ -2362,7 +2594,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2398,7 +2630,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2415,88 +2647,176 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
}
+/* ----------
+ * snapshot_statentry() -
+ *
+ * Common routine for functions pgstat_fetch_stat_*entry()
+ *
+ * Returns the pointer to the snapshot of the shared entry for the key or NULL
+ * if not found. Returned snapshots are stable during the current transaction
+ * or until pgstat_clear_snapshot() is called.
+ *
+ * Created snapshots are stored in pgStatSnapshotHash.
+ */
+static void *
+snapshot_statentry(const PgStatTypes type, const Oid dbid, const Oid objid)
+{
+ PgStatSnapshot *snap = NULL;
+ bool found;
+ PgStatHashEntryKey key;
+ size_t statentsize = pgstat_entsize[type];
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ /*
+ * Create new hash, with rather arbitrary initial number of entries since
+ * we don't know how this hash will grow.
+ */
+ if (!pgStatSnapshotHash)
+ {
+ HASHCTL ctl;
+
+ /*
+ * Create the hash in the stats context
+ *
+ * The entry is prepended by common header part represented by
+ * PgStatSnapshot.
+ */
+
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = PgStatSnapshotSize(statentsize);
+ ctl.hcxt = pgStatSnapshotContext;
+ pgStatSnapshotHash = hash_create("pgstat snapshot hash", 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ /* Find a snapshot */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ snap = hash_search(pgStatSnapshotHash, &key, HASH_ENTER, &found);
+
+ /*
+ * Refer shared hash if not found in the snapshot hash.
+ *
+ * In transaction state, it is obvious that we should create a snapshot
+ * entriy for consistency. If we are not, we return an up-to-date entry.
+ * Having said that, we need a snapshot since shared stats entry can be
+ * modified anytime. We share the same snapshot entry for the purpose.
+ */
+ if (!found || !IsTransactionState())
+ {
+ PgStatEnvelope *shenv;
+
+ shenv = get_stat_entry(type, dbid, objid, true, NULL, NULL);
+
+ if (shenv)
+ memcpy(&snap->body, &shenv->body, statentsize);
+
+ snap->negative = !shenv;
+ }
+
+ if (snap->negative)
+ return NULL;
+
+ return &snap->body;
+}
+
+
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *)
+ snapshot_statentry(PGSTAT_TYPE_DB, dbid, InvalidOid);
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_snapshot() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_snapshot(bool shared, Oid reloid)
+{
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(PGSTAT_TYPE_TABLE, dboid, reloid);
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2510,24 +2830,48 @@ pgstat_fetch_stat_tabentry(Oid relid)
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
-
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
-
- return funcentry;
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, func_id);
}
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ * ----------
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+
+ attach_shared_stats();
+
+ /* Nothing to do if already done */
+ if (global_snapshot_is_valid)
+ return;
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ memcpy(&snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ LWLockRelease(StatsLock);
+
+ global_snapshot_is_valid = true;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
+}
/* ----------
* pgstat_fetch_stat_beentry() -
@@ -2599,9 +2943,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return &snapshot_archiverStats;
}
@@ -2616,9 +2961,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return &snapshot_globalStats;
}
@@ -2832,8 +3178,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3009,12 +3355,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3027,7 +3376,7 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
@@ -3044,6 +3393,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3304,7 +3655,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3599,9 +3951,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
break;
@@ -4230,94 +4579,71 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Report archiver statistics
* ----------
*/
void
-pgstat_send_archiver(const char *xlog, bool failed)
+pgstat_report_archiver(const char *xlog, bool failed)
{
- PgStat_MsgArchiver msg;
+ TimestampTz now = GetCurrentTimestamp();
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ if (failed)
+ {
+ /* Failed archival attempt */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ /* Successful archival operation */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *l = &BgWriterStats;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += l->timed_checkpoints;
+ shared_globalStats->requested_checkpoints += l->requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += l->checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += l->checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += l->buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += l->buf_written_clean;
+ shared_globalStats->maxwritten_clean += l->maxwritten_clean;
+ shared_globalStats->buf_written_backend += l->buf_written_backend;
+ shared_globalStats->buf_fsync_backend += l->buf_fsync_backend;
+ shared_globalStats->buf_alloc += l->buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4326,425 +4652,30 @@ pgstat_send_bgwriter(void)
}
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
-{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
-
- return result;
-}
-
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
-{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->inserts_since_vacuum = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
-}
-
-
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
elog(DEBUG2, "writing stats file \"%s\"", statfile);
+ create_missing_dbentries();
+
/*
* Open the statistics temp file to write out the current values.
*/
@@ -4761,7 +4692,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4773,32 +4704,31 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_DB, InvalidOid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &(*penv)->body;
+
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_database_stats(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
@@ -4809,6 +4739,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
+
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4841,55 +4773,19 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
}
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
+ * pgstat_write_database_stats() -
+ * Write the stat file for a single database.
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_database_stats(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -4897,8 +4793,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -4925,24 +4821,31 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_TABLE, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatTabEntry *tabentry = (PgStat_StatTabEntry *) &(*penv)->body;
+
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_FUNCTION, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatFuncEntry *funcentry =
+ (PgStat_StatFuncEntry *) &(*penv)->body;
+
fputc('F', fpout);
rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* No more output to be done. Close the temp file and replace the old
@@ -4976,94 +4879,165 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
+}
- if (permanent)
+/* ----------
+ * create_missing_dbentries() -
+ *
+ * There may be the case where database entry is missing for the database
+ * where object stats are recorded. This function creates such missing
+ * dbentries so that so that all stats entries can be written out to files.
+ * ----------
+ */
+static void
+create_missing_dbentries(void)
+{
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ HTAB *oidhash;
+ HASHCTL ctl;
+ HASH_SEQ_STATUS scan;
+ Oid *poid;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = CurrentMemoryContext;
+ oidhash = hash_create("Temporary table of OIDs",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Collect OID from the shared stats hash */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ hash_search(oidhash, &p->key.databaseid, HASH_ENTER, NULL);
+ dshash_seq_term(&hstat);
+
+ /* Create missing database entries if not exists. */
+ hash_seq_init(&scan, oidhash);
+ while ((poid = (Oid *) hash_seq_search(&scan)) != NULL)
+ (void) get_stat_entry(PGSTAT_TYPE_DB, *poid, InvalidOid,
+ false, init_dbentry, NULL);
+
+ hash_destroy(oidhash);
+}
+
+
+/* ----------
+ * get_stat_entry() -
+ *
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new envelope. If found is not NULL, it is set to true if
+ * existing entry is found or false if not.
+ * ----------
+ */
+static PgStatEnvelope *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait, entry_initializer initfunc, bool *found)
+{
+ bool create = (initfunc != NULL);
+ PgStatHashEntry *shent;
+ PgStatEnvelope *shenv = NULL;
+ PgStatHashEntryKey key;
+ bool myfound;
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ shent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &myfound);
+ if (shent)
{
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
+ if (create && !myfound)
+ {
+ /* Create new stats envelope. */
+ size_t envsize = PgStatEnvelopeSize(pgstat_entsize[type]);
+ dsa_pointer chunk = dsa_allocate0(area, envsize);
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
+ shenv = dsa_get_address(area, chunk);
+ shenv->type = type;
+ shenv->databaseid = dbid;
+ shenv->objectid = objid;
+ shenv->len = pgstat_entsize[type];
+ LWLockInitialize(&shenv->lock, LWTRANCHE_STATS);
+
+ /*
+ * The lock on dshsh is released just after. Call initializer
+ * callback before it is exposed to other process.
+ */
+ if (initfunc)
+ initfunc(shenv);
+
+ /* Link the new entry from the hash entry. */
+ shent->env = chunk;
+ }
+ else
+ shenv = dsa_get_address(area, shent->env);
+
+ dshash_release_lock(pgStatSharedHash, shent);
}
+
+ if (found)
+ *found = myfound;
+
+ return shenv;
}
+
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * Reads in existing activity statistics files into the shared stats hash.
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
+ PgStatEnvelope *env;
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5072,7 +5046,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5080,38 +5054,30 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5125,7 +5091,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
fpin) != offsetof(PgStat_StatDBEntry, tables))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5134,76 +5100,33 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+
+ env = get_stat_entry(PGSTAT_TYPE_DB, dbbuf.databaseid,
+ InvalidOid,
+ false, init_dbentry, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ /* don't allow duplicate dbentries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, tables));
+ /* Read the data from the database-specific file. */
+ pgstat_read_db_statsfile(dbentry);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5213,59 +5136,50 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
/* ----------
* pgstat_read_db_statsfile() -
*
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry)
{
+ PgStatEnvelope *env;
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
FILE *fpin;
int32 format_id;
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5278,14 +5192,14 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5298,25 +5212,21 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE,
+ dbentry->databaseid, tabbuf.tableid,
+ false, init_tabentry, &found);
+ tabentry = (PgStat_StatTabEntry *) &env->body;
+ /* don't allow duplicate entries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5332,25 +5242,20 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE, dbentry->databaseid,
+ funcbuf.functionid,
+ false, init_funcentry, &found);
+ funcentry = (PgStat_StatFuncEntry *) &env->body;
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5366,7 +5271,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5374,292 +5279,15 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
-done:
FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
-}
-
-/* ----------
- * pgstat_setup_memcxt() -
- *
- * Create pgStatLocalContext, if not already done.
- * ----------
- */
-static void
-pgstat_setup_memcxt(void)
-{
- if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
@@ -5678,756 +5306,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum = tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- tabentry->inserts_since_vacuum = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum += tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * It is quite possible that a non-aggressive VACUUM ended up skipping
- * various pages, however, we'll zero the insert counter here regardless.
- * It's currently used only to track when we need to perform an
- * "insert" autovacuum, which are mainly intended to freeze newly inserted
- * tuples. Zeroing this may just mean we'll not try to vacuum the table
- * again until enough tuples have been inserted to trigger another insert
- * autovacuum. An anti-wraparound autovacuum will catch any persistent
- * stragglers.
- */
- tabentry->inserts_since_vacuum = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
}
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
+ if (pgStatSnapshotContext)
{
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
+ MemoryContextReset(pgStatSnapshotContext);
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
+ /* Reset variables that pointed to the context */
+ global_snapshot_is_valid = false;
+ pgStatSnapshotHash = NULL;
}
}
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
-}
-
/*
* Convert a potentially unsafely truncated activity string (see
* PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a4b9d212a2..51d02b6cd7 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -255,7 +255,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -503,7 +502,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1326,12 +1324,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1780,11 +1772,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2694,8 +2681,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3058,8 +3043,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3126,13 +3109,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3205,22 +3181,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3681,22 +3641,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3892,8 +3836,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3917,8 +3859,7 @@ PostmasterStateMachine(void)
{
/*
* PM_WAIT_DEAD_END state ends when the BackendList is entirely empty
- * (ie, no dead_end children remain), and the archiver and stats
- * collector are gone too.
+ * (ie, no dead_end children remain), and the archiveris gone too.
*
* The reason we wait for those two is to protect them against a new
* postmaster starting conflicting subprocesses; this isn't an
@@ -3928,8 +3869,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4130,8 +4070,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5109,18 +5047,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5239,12 +5165,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6139,7 +6059,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6195,8 +6114,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6431,7 +6348,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index a2e28b064c..261920b961 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1445,8 +1445,8 @@ is_checksummed_file(const char *fullpath, const char *filename)
*
* If 'missing_ok' is true, will not throw an error if the file is not found.
*
- * If dboid is anything other than InvalidOid then any checksum failures detected
- * will get reported to the stats collector.
+ * If dboid is anything other than InvalidOid then any checksum failures
+ * detected will get reported to the activity stats facility.
*
* Returns true if the file was successfully sent, false if 'missing_ok',
* and the file did not exist.
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index e72d607a23..ea5030452d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1948,7 +1948,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ BgWriterStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2058,7 +2058,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2248,7 +2248,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2256,7 +2256,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 427b0d59cd..58a442f482 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 4c14e51c67..f61fd3e8ad 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -523,6 +523,7 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
+ LWLockRegisterTranche(LWTRANCHE_STATS, "activity_statistics");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index db47843229..97eccb35d3 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -49,3 +49,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
LogicalRepWorkerLock 43
CLogTruncationLock 44
+StatsLock 45
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 360b5bf5bf..c10d80f035 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -362,10 +362,10 @@ smgrdounlink(SMgrRelation reln, bool isRedo)
DropRelFileNodesAllBuffers(&rnode, 1);
/*
- * It'd be nice to tell the stats collector to forget it immediately, too.
- * But we can't because we don't know the OID (and in cases involving
- * relfilenode swaps, it's not always clear which table OID to forget,
- * anyway).
+ * It'd be nice to tell the activity stats facility to forget it
+ * immediately, too. But we can't because we don't know the OID (and in
+ * cases involving relfilenode swaps, it's not always clear which table OID
+ * to forget, anyway).
*/
/*
@@ -435,8 +435,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
DropRelFileNodesAllBuffers(rnodes, nrels);
/*
- * It'd be nice to tell the stats collector to forget them immediately,
- * too. But we can't because we don't know the OIDs.
+ * It'd be nice to tell the activity stats facility to forget them
+ * immediately, too. But we can't because we don't know the OIDs.
*/
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5b677863b9..3f906210c2 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3192,6 +3192,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3766,6 +3772,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4168,11 +4175,12 @@ PostgresMain(int argc, char *argv[],
* Note: this includes fflush()'ing the last of the prior output.
*
* This is also a good time to send collected statistics to the
- * collector, and to update the PS stats display. We avoid doing
- * those every time through the message loop because it'd slow down
- * processing of batched messages, and because we don't want to report
- * uncommitted updates (that confuses autovacuum). The notification
- * processor wants a call too, if we are not in a transaction block.
+ * activity statistics, and to update the PS stats display. We avoid
+ * doing those every time through the message loop because it'd slow
+ * down processing of batched messages, and because we don't want to
+ * report uncommitted updates (that confuses autovacuum). The
+ * notification processor wants a call too, if we are not in a
+ * transaction block.
*/
if (send_ready_for_query)
{
@@ -4204,6 +4212,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4216,8 +4226,13 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4252,7 +4267,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4260,6 +4275,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6d66ff8b44..5f80fa755b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1,7 +1,7 @@
/*-------------------------------------------------------------------------
*
* pgstatfuncs.c
- * Functions for accessing the statistics collector data
+ * Functions for accessing the activity statistics data
*
* Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -33,9 +33,6 @@
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1260,7 +1257,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1276,7 +1273,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1292,7 +1289,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1308,7 +1305,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1324,7 +1321,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1340,7 +1337,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1356,7 +1353,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1372,7 +1369,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1388,7 +1385,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1421,7 +1418,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1437,7 +1434,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1452,7 +1449,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1467,7 +1464,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1482,7 +1479,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1497,7 +1494,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1512,7 +1509,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1527,11 +1524,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1546,7 +1543,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1564,7 +1561,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1601,7 +1598,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1617,7 +1614,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..51748c99ad 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index a7b7b12249..e6b6126141 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -232,9 +232,6 @@ GetBackendTypeDesc(BackendType backendType)
case B_ARCHIVER:
backendDesc = "archiver";
break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
case B_LOGGER:
backendDesc = "logger";
break;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index f4247ea70d..f65d05c24c 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -631,6 +632,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1241,6 +1244,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 79bc7ac8ca..377bc43132 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -745,8 +745,8 @@ const char *const config_group_names[] =
gettext_noop("Statistics"),
/* STATS_MONITORING */
gettext_noop("Statistics / Monitoring"),
- /* STATS_COLLECTOR */
- gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* STATS_ACTIVITY */
+ gettext_noop("Statistics / Query and Index Activity Statistics"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
@@ -1476,7 +1476,7 @@ static struct config_bool ConfigureNamesBool[] =
#endif
{
- {"track_activities", PGC_SUSET, STATS_COLLECTOR,
+ {"track_activities", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects information about executing commands."),
gettext_noop("Enables the collection of information on the currently "
"executing command of each session, along with "
@@ -1487,7 +1487,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_counts", PGC_SUSET, STATS_COLLECTOR,
+ {"track_counts", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects statistics on database activity."),
NULL
},
@@ -1496,7 +1496,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_io_timing", PGC_SUSET, STATS_COLLECTOR,
+ {"track_io_timing", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects timing statistics for database I/O activity."),
NULL
},
@@ -4588,7 +4588,7 @@ static struct config_enum ConfigureNamesEnum[] =
},
{
- {"track_functions", PGC_SUSET, STATS_COLLECTOR,
+ {"track_functions", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects function-level statistics on database activity."),
NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e9f8ca775d..970214f275 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -565,7 +565,7 @@
# STATISTICS
#------------------------------------------------------------------------------
-# - Query and Index Statistics Collector -
+# - Query and Index Activity Statistics -
#track_activities = on
#track_counts = on
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..927ae319b1 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 106;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 619b2f9c71..23984d6e24 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,8 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -320,7 +322,6 @@ typedef enum BackendType
B_WAL_SENDER,
B_WAL_WRITER,
B_ARCHIVER,
- B_STATS_COLLECTOR,
B_LOGGER,
} BackendType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 763c1ee2bd..69bd794806 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -15,9 +15,11 @@
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,33 +43,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -78,9 +53,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -111,18 +85,17 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_delta_live_tuples;
PgStat_Counter t_delta_dead_tuples;
PgStat_Counter t_changed_tuples;
+ bool reset_changed_tuples;
PgStat_Counter t_blocks_fetched;
PgStat_Counter t_blocks_hit;
+
+ PgStat_Counter vacuum_count;
+ PgStat_Counter autovac_vacuum_count;
+ PgStat_Counter analyze_count;
+ PgStat_Counter autovac_analyze_count;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -156,6 +129,10 @@ typedef struct PgStat_TableStatus
Oid t_id; /* table's OID */
bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
} PgStat_TableStatus;
@@ -181,280 +158,32 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgTempFile
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter buf_alloc;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -486,96 +215,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -584,13 +225,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9D
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -600,7 +237,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -610,29 +246,55 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ Oid databaseid;
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/* ----------
+ * PgStat_HashMountInfo dshash mount information
+ * ----------
+ */
+typedef struct PgStat_HashMountInfo
+{
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
+} PgStat_HashMountInfo;
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
Oid tableid;
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -652,19 +314,15 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -679,7 +337,7 @@ typedef struct PgStat_StatFuncEntry
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -695,7 +353,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -761,7 +419,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1005,7 +662,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1202,13 +859,15 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1223,29 +882,26 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
@@ -1406,8 +1062,8 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1416,11 +1072,15 @@ extern void pgstat_send_bgwriter(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_snapshot(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 8fda8e4f78..13371e8cb7 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -220,6 +220,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_SXACT,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index 454c2df487..2c65231b04 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -88,7 +88,7 @@ enum config_group
PROCESS_TITLE,
STATS,
STATS_MONITORING,
- STATS_COLLECTOR,
+ STATS_ACTIVITY,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 83a15f6795..77d1572a99 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.18.2
* [PATCH v25 6/8] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 500ms and such large data is
serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 500ms. Locks
on the shared statistics is acquired by the units of such like tables,
functions so the expected chance of collision are not so
high. Furthermore, until 1 second has elapsed since the last flushing
to shared stats, lock failure postpones stats flushing so that lock
contention doesn't slow down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/transam/xlog.c | 4 +-
src/backend/catalog/index.c | 24 +-
src/backend/postmaster/autovacuum.c | 12 +-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 12 +-
src/backend/postmaster/pgarch.c | 4 +-
src/backend/postmaster/pgstat.c | 4625 +++++++-----------
src/backend/postmaster/postmaster.c | 85 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 1 +
src/backend/tcop/postgres.c | 26 +-
src/backend/utils/adt/pgstatfuncs.c | 53 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 +
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 2 +
src/include/pgstat.h | 500 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/timeout.h | 1 +
20 files changed, 1991 insertions(+), 3387 deletions(-)
inline patch 1: src/backend/access/transam/xlog.c, src/backend/catalog/index.c, src/backend/postmaster/autovacuum.c and 17 more (+1991/-3387 lines) download inline-1.patch
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 5c477211e9..4ea29b8997 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8506,9 +8506,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..613cef9282 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1687,28 +1687,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be sent by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Close relations */
table_close(pg_class, RowExclusiveLock);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index da75e755f0..333712d3c5 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -1956,15 +1956,15 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
+ /* Start a transaction so our commands have one to play into. */
+ StartTransactionCommand();
+
/*
* may be NULL if we couldn't find an entry (only happens if we are
* forcing a vacuum for anti-wrap purposes).
*/
dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- /* Start a transaction so our commands have one to play into. */
- StartTransactionCommand();
-
/*
* Clean up any dead statistics collector entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
@@ -2748,12 +2748,10 @@ get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
if (isshared)
{
if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(shared, relid);
}
else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(dbentry, relid);
return tabentry;
}
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..94bdd664b5 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -236,7 +236,7 @@ BackgroundWriterMain(void)
/*
* Send off activity statistics to the stats collector
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..8a2fd0ddb2 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -350,7 +350,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ BgWriterStats.requested_checkpoints++;
}
/*
@@ -364,7 +364,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ BgWriterStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -492,7 +492,7 @@ CheckpointerMain(void)
* worth the trouble to split the stats support into two independent
* stats message types.)
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* Sleep until we are signaled or it's time for another checkpoint or
@@ -693,7 +693,7 @@ CheckpointWriteDelay(int flags, double progress)
/*
* Report interim activity statistics to the stats collector.
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1238,8 +1238,8 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ BgWriterStats.buf_written_backend += CheckpointerShmem->num_backend_writes;
+ BgWriterStats.buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 6fe7a136ba..f0b524ca50 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -401,7 +401,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we successfully
* archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
@@ -411,7 +411,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we failed to
* archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index f9287b7942..34a4005791 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,23 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects per-table and per-function usage statistics of all backends on
+ * shared memory. pg_count_*() and friends are the interface to locally store
+ * backend activities during a transaction. Then pgstat_flush_stat() is called
+ * at the end of a transaction to publish the local stats on shared memory.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, we update shared stats no more
+ * often than intervals of PGSTAT_STAT_MIN_INTERVAL(500ms). In the case where
+ * all the local numbers cannot be flushed immediately, we postpone updates
+ * and try the next chance after the interval of
+ * PGSTAT_STAT_RETRY_INTERVAL(100ms), but we don't wait for no longer than
+ * PGSTAT_STAT_MAX_INTERVAL(1000ms).
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -19,18 +27,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,68 +36,43 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
-
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
+#define PGSTAT_STAT_MIN_INTERVAL 500 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_STAT_RETRY_INTERVAL 100 /* Retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_STAT_MAX_INTERVAL 1000 /* Longest interval of stats data
+ * updates; in milliseconds. */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -116,7 +87,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -131,76 +101,96 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
-/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-
-/* ----------
- * Local data
- * ----------
- */
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
-
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
-
-/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
- *
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
- */
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-
-typedef struct TabStatusArray
+/* Shared stats bootstrap information, protected by StatsLock */
+typedef struct StatsShmemStruct
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ dsa_handle stats_dsa_handle; /* DSA handle for stats data */
+ dshash_table_handle db_hash_handle;
+ dsa_pointer global_stats;
+ dsa_pointer archiver_stats;
+ int refcount;
+} StatsShmemStruct;
+
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
+
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
+static dshash_table *pgStatDBHash = NULL;
+
+
+/* parameter for shared hashes */
+static const dshash_parameters dsh_dbparams = {
+ sizeof(Oid),
+ SHARED_DBENT_SIZE,
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+static const dshash_parameters dsh_tblparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatTabEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+static const dshash_parameters dsh_funcparams = {
+ sizeof(Oid),
+ sizeof(PgStat_StatFuncEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * Backends store per-table info that's waiting to be flushed out to shared
+ * memory in this hash table (indexed by table OID).
*/
-typedef struct TabStatHashEntry
-{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+static HTAB *pgStatTables = NULL;
/*
- * Hash table for O(1) t_id -> tsa_entry lookup
+ * Backends store per-function info that's waiting to be flushed out to shared
+ * memory in this hash table (indexed by function OID).
*/
-static HTAB *pgStatTabHash = NULL;
+static HTAB *pgStatFunctions = NULL;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Backends store database-wide counters that's waiting to be flushed out to
+ * shared memory.
*/
-static HTAB *pgStatFunctions = NULL;
+static PgStat_TableCounts pgStatMyDatabaseStats = {0};
+static PgStat_TableCounts pgStatSharedDatabaseStats = {0};
/*
* Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * written out to the shared stats.
*/
+static bool have_mydatabase_stats = false;
+static bool have_shdatabase_stats = false;
+static bool have_table_stats = false;
static bool have_function_stats = false;
+/* common header of snapshot entry in reader snapshot hash */
+typedef struct PgStat_snapshot
+{
+ Oid key;
+ bool negative;
+ void *body; /* end of header part: to keep alignment */
+} PgStat_snapshot;
+
+/* Hash entry struct for checksum_failures above */
+typedef struct ChecksumFailureEnt
+{
+ Oid dboid;
+ int count;
+} ChecksumFailureEnt;
+
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
* into PgStat_TableStatus counters until we know if it is going to commit
@@ -236,11 +226,15 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatLocalHash = NULL;
+static bool clear_snapshot = false;
+
+/* Count checksum failure for each database */
+HTAB *checksum_failures = NULL;
+int nchecksum_failures = 0;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -249,19 +243,17 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
- */
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are collected not per database nor per table
+ * basis. shared_* points to shared memory and snapshot_* are backend
+ * snapshots. Their validity is indicated by global_snapshot_is_valid.
*/
-static List *pending_write_requests = NIL;
+static bool global_snapshot_is_valid = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats snapshot_globalStats;
/*
* Total time charged to functions so far in the current backend.
@@ -275,33 +267,34 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
+static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool exclusive,
+ bool nowait, bool create);
+static PgStat_StatTabEntry *pgstat_get_tab_entry(dshash_table *table,
Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static void pgstat_write_pgStatDBHashfile(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_pgStatDBHashfile(PgStat_StatDBEntry *dbentry);
static void pgstat_read_current_status(void);
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
+static void pgstat_flush_dbstats(bool shared, bool nowait);
+static bool pgstat_flush_tabstats(Oid dbid, dshash_table_handle tabhandle,
+ bool nowait);
+static bool pgstat_flush_funcstats(dshash_table_handle funchandle, bool nowait);
+static bool pgstat_update_tabentry(dshash_table *tabhash,
+ PgStat_TableStatus *stat, bool nowait);
static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
+static void pgstat_remove_useless_entries(const dshash_table_handle dshhandle,
+ const dshash_parameters *dshparams,
+ HTAB *oidtab);
static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
static void pgstat_setup_memcxt(void);
+static void pgstat_flush_checksum_failure(bool nowait);
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry_snapshot(PgStat_StatDBEntry *dbent, Oid funcid);
+static void pgstat_snapshot_global_stats(void);
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
@@ -309,484 +302,210 @@ static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
+/* ------------------------------------------------------------
+ * Local support functions follow
+ * ------------------------------------------------------------
+ */
+static void reset_dbentry_counters(PgStat_StatDBEntry *dbentry);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
+ bool found;
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
- }
+ Assert(!found);
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ }
+}
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
+/* ----------
+ * pgstat_attach_shared_stats() -
+ *
+ * Attach shared or create stats memory.
+ * ---------
+ */
+static void
+pgstat_attach_shared_stats(void)
+{
+ PgStat_StatDBEntry *dbent;
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+ MemoryContext oldcontext;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * Don't use dsm under postmaster, or when not tracking counts.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ pgstat_setup_memcxt();
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ if (area)
+ return;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
+ {
+ /* Need to create shared memory area and load saved stats if any. */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatDBHash = dshash_create(area, &dsh_dbparams, 0);
- test_byte++; /* just make sure variable is changed */
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->db_hash_handle = dshash_get_hash_table_handle(pgStatDBHash);
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Load saved data if any. */
+ pgstat_read_statsfiles();
- /* If we get here, we have a working socket */
- break;
+ StatsShmem->refcount = 1;
}
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockRelease(StatsLock);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * If we're not the first process, attach existing shared stats area
+ * outside StatsLock.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (!area)
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
- }
-
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
+ /* Shared area already exists. Just attach it. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatDBHash = dshash_attach(area, &dsh_dbparams,
+ StatsShmem->db_hash_handle, 0);
+
+ /* Setup local variables */
+ pgStatLocalHash = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
}
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
-
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
+ MemoryContextSwitchTo(oldcontext);
/*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
+ * create db entries for the current database and shared table if not
+ * created yet.
*/
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ dbent = pgstat_get_db_entry(MyDatabaseId, false, false, true);
+ Assert(dbent);
+ dshash_release_lock(pgStatDBHash, dbent);
+ dbent = pgstat_get_db_entry(InvalidOid, false, false, true);
+ Assert(dbent);
+ dshash_release_lock(pgStatDBHash, dbent);
+
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+ global_snapshot_is_valid = false;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * pgstat_detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+pgstat_detach_shared_stats(bool write_stats)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
+ /* write out the shared stats to file if needed */
+ if (--StatsShmem->refcount < 1)
{
- int nchars;
- Oid tmp_oid;
+ if (write_stats)
+ pgstat_write_statsfiles();
- /*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
- */
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
+ /* We're the last process. Invalidate the dsa area handle. */
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ }
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
+ LWLockRelease(StatsLock);
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
- }
- FreeDir(dir);
+ /*
+ * Detach the area. Automatically destroyed when the last process detached
+ * it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatDBHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatLocalHash = NULL;
+ global_snapshot_is_valid = false;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
-
-#ifdef EXEC_BACKEND
-
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
/*
- * Okay, fork off the collector.
+ * We could directly remove files and recreate the shared memory area. But
+ * detach then attach for simplicity.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
+ pgstat_detach_shared_stats(false); /* Don't write */
+ pgstat_attach_shared_stats();
}
/* ------------------------------------------------------------
@@ -794,259 +513,441 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_STAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_STAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_STAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
- int i;
+ PgStat_StatDBEntry *dbent;
+ bool nowait = !force; /* Don't use force ever after */
+ long elapsed;
+ long secs;
+ int usecs;
+ dshash_table_handle tables_handle;
+ dshash_table_handle functions_handle;
+ bool process_shared_tables = false;
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (area == NULL ||
+ (!have_table_stats && !have_function_stats &&
+ !have_mydatabase_stats && !have_shdatabase_stats &&
+ pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
+ checksum_failures != NULL))
+ return 0;
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
-
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
- /*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
- */
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ if (nowait)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /*
+ * Don't flush stats unless it's the time. Returns time to wait in
+ * milliseconds.
+ */
+ if (now < next_flush)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
+ /* Record the oldest pending update if not yet. */
+ if (pending_since == 0)
+ pending_since = now;
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
+ /* now < next_flush here */
+ return (next_flush - now) / 1000;
+ }
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
- continue;
+ /*
+ * Don't keep pending updates longer than PGSTAT_STAT_MAX_INTERVAL.
+ */
+ if (pending_since > 0)
+ {
+ TimestampDifference(pending_since, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs / 1000;
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
- {
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
- }
+ if (elapsed > PGSTAT_STAT_MAX_INTERVAL)
+ nowait = false;
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
}
- /*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
- */
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
-
- /* Now, send function statistics */
- pgstat_send_funcstats();
-}
+ /* Flush out individual stats tables */
+ dbent = pgstat_get_db_entry(MyDatabaseId, false, nowait, false);
+ tables_handle = dbent->tables;
+ functions_handle = dbent->functions;
+ dshash_release_lock(pgStatDBHash, dbent);
-/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
- */
+ /* dbent is no longer usable but indicates it was acquired */
+ if (dbent)
+ {
+ process_shared_tables =
+ pgstat_flush_tabstats(MyDatabaseId, tables_handle, nowait);
+ pgstat_flush_funcstats(functions_handle, nowait);
+ }
+ else
+ {
+ /* uncertain whether shared table stats exists, try it */
+ process_shared_tables = true;
+ }
+
+ /* update database-side stats */
+ pgstat_flush_checksum_failure(nowait);
+ pgstat_flush_dbstats(false, nowait); /* MyDatabase */
+
+ if (process_shared_tables)
+ {
+ /* shared tables found, process them */
+ dbent = pgstat_get_db_entry(InvalidOid, false, nowait, false);
+ tables_handle = dbent->tables;
+ dshash_release_lock(pgStatDBHash, dbent);
+
+ if (dbent)
+ pgstat_flush_tabstats(InvalidOid, tables_handle, nowait);
+ }
+ pgstat_flush_dbstats(true, nowait); /* Shared tables */
+
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (shared_globalStats->stats_timestamp < now)
+ shared_globalStats->stats_timestamp = now;
+ LWLockRelease(StatsLock);
+
+ /* Record how long we are keeping pending updates. */
+ if (have_table_stats || have_function_stats ||
+ have_mydatabase_stats || have_shdatabase_stats ||
+ checksum_failures != NULL)
+ {
+ /* Preserve the first value */
+ if (pending_since == 0)
+ pending_since = now;
+
+ /*
+ * It's possible that the retry interval is longer than the limit by
+ * PGSTAT_STAT_MAX_INTERVAL. We don't bother that since it's not so
+ * much.
+ */
+ return PGSTAT_STAT_RETRY_INTERVAL;
+ }
+
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_STAT_MIN_INTERVAL * 1000;
+ pending_since = 0;
+
+ return 0;
+}
+
+
+/*
+ * pgstat_flush_dbstats: Flushes database stats out to shared statistics.
+ *
+ * If nowait is true, returns immediately if required lock was not acquired.
+ */
static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+pgstat_flush_dbstats(bool shared, bool nowait)
{
- int n;
- int len;
+ PgStat_StatDBEntry *dbent;
+ PgStat_TableCounts *s;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ if (shared)
+ {
+ if (!have_shdatabase_stats)
+ return;
+ dbent = pgstat_get_db_entry(InvalidOid, false, nowait, false);
+ if (!dbent)
+ return;
- /*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
- */
- if (OidIsValid(tsmsg->m_databaseid))
+ s = &pgStatSharedDatabaseStats;
+ have_shdatabase_stats = false;
+ }
+ else
+ {
+ if (!have_mydatabase_stats)
+ return;
+ dbent = pgstat_get_db_entry(MyDatabaseId, false, nowait, false);
+ if (!dbent)
+ return;
+
+ s = &pgStatMyDatabaseStats;
+ have_mydatabase_stats = false;
+ }
+
+ /* We got the database entry, update database-wide stats */
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
+ dbent->counts.n_tuples_returned += s->t_tuples_returned;
+ dbent->counts.n_tuples_fetched += s->t_tuples_fetched;
+ dbent->counts.n_tuples_inserted += s->t_tuples_inserted;
+ dbent->counts.n_tuples_updated += s->t_tuples_updated;
+ dbent->counts.n_tuples_deleted += s->t_tuples_deleted;
+ dbent->counts.n_blocks_fetched += s->t_blocks_fetched;
+ dbent->counts.n_blocks_hit += s->t_blocks_hit;
+
+ if (!shared)
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ dbent->counts.n_xact_commit += pgStatXactCommit;
+ dbent->counts.n_xact_rollback += pgStatXactRollback;
+ dbent->counts.n_block_read_time += pgStatBlockReadTime;
+ dbent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
pgStatBlockWriteTime = 0;
}
- else
+ LWLockRelease(&dbent->lock);
+
+ dshash_release_lock(pgStatDBHash, dbent);
+}
+
+/*
+ * pgstat_flush_tabstats: Flushes table stats out to shared statistics.
+ *
+ * If nowait is true, returns false if required lock was not acquired
+ * immediately. In that case, unapplied table stats updates are left alone in
+ * pgStatTables to wait for the next chance. cxt holds some dshash related
+ * values that we want to carry around while updating shared stats.
+ *
+ * Returns true if entries for another database is found in pgStatTables.
+ */
+static bool
+pgstat_flush_tabstats(Oid dbid, dshash_table_handle tabhandle, bool nowait)
+{
+ static const PgStat_TableCounts all_zeroes;
+
+ HASH_SEQ_STATUS scan;
+ PgStat_TableStatus *bestat;
+ dshash_table *tabhash;
+ bool anotherdb_found = false;
+
+ /* nothing to do, just return */
+ if (!have_table_stats)
+ return false;
+
+ have_table_stats = false;
+
+ tabhash = dshash_attach(area, &dsh_tblparams, tabhandle, 0);
+
+ /*
+ * Scan through the pgStatTables to find tables that actually have counts,
+ * and try flushing it out to shared stats.
+ */
+ hash_seq_init(&scan, pgStatTables);
+ while ((bestat = (PgStat_TableStatus *) hash_seq_search(&scan)) != NULL)
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ bool remove_entry = false;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&bestat->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ remove_entry = true;
+ /* Ignore entries of databases other than our current target */
+ else if (dbid != (bestat->t_shared ? InvalidOid : MyDatabaseId))
+ anotherdb_found = true;
+ else if (pgstat_update_tabentry(tabhash, bestat, nowait))
+ {
+ PgStat_TableCounts *s;
+
+ if (dbid == bestat->t_shared)
+ {
+ s = &pgStatSharedDatabaseStats;
+ have_shdatabase_stats = true;
+ }
+ else
+ {
+ s = &pgStatMyDatabaseStats;
+ have_mydatabase_stats = true;
+ }
+
+ /* database count is applied at once later */
+ s->t_tuples_returned += bestat->t_counts.t_tuples_returned;
+ s->t_tuples_fetched += bestat->t_counts.t_tuples_fetched;
+ s->t_tuples_inserted += bestat->t_counts.t_tuples_inserted;
+ s->t_tuples_updated += bestat->t_counts.t_tuples_updated;
+ s->t_tuples_deleted += bestat->t_counts.t_tuples_deleted;
+ s->t_blocks_fetched += bestat->t_counts.t_blocks_fetched;
+ s->t_blocks_hit += bestat->t_counts.t_blocks_hit;
+
+ remove_entry = true;
+ }
+
+ if (remove_entry)
+ {
+ /*
+ * Reuse of the entry is detected with t_id in pgstat_initstats.
+ * Set invalid value after removal because the value is needed to
+ * remove the entry.
+ */
+ hash_search(pgStatTables, &bestat->t_id, HASH_REMOVE, NULL);
+ bestat->t_id = InvalidOid;
+ }
+ else
+ have_table_stats = true;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ dshash_detach(tabhash);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return anotherdb_found;
}
+
/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+ * pgstat_flush_funcstats: Flushes function stats.
+ *
+ * If nowait is true, returns false on lock failure. Unapplied local hash
+ * entries are left alone.
+ *
+ * Returns true if some entries are left unflushed.
*/
-static void
-pgstat_send_funcstats(void)
+static bool
+pgstat_flush_funcstats(dshash_table_handle funchandle, bool nowait)
{
/* we assume this inits to all zeroes: */
static const PgStat_FunctionCounts all_zeroes;
+ HASH_SEQ_STATUS scan;
+ PgStat_BackendFunctionEntry *bestat;
+ dshash_table *funchash = NULL;
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
+ /* nothing to do, just return */
if (pgStatFunctions == NULL)
- return;
+ return false;
+
+ have_function_stats = false;
+
+ /* dshash for function stats is created on-demand */
+ if (funchandle == DSM_HANDLE_INVALID)
+ {
+ PgStat_StatDBEntry *dbent =
+ pgstat_get_db_entry(MyDatabaseId, false, false, false);
+
+ Assert(dbent);
+
+ funchash = dshash_create(area, &dsh_funcparams, 0);
+
+ LWLockAcquire(&dbent->lock, LW_EXCLUSIVE);
+ if (dbent->functions == DSM_HANDLE_INVALID)
+ funchandle = dbent->functions =
+ dshash_get_hash_table_handle(funchash);
+ else
+ {
+ /* someone else simultaneously created it, discard mine. */
+ dshash_destroy(funchash);
+ funchandle = dbent->functions;
+ }
+ LWLockRelease(&dbent->lock);
+
+ /* dbent is no longer needed, release it right now */
+ dshash_release_lock(pgStatDBHash, dbent);
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
+ if (funchash == NULL)
+ funchash = dshash_attach(area, &dsh_funcparams, funchandle, 0);
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ /*
+ * Scan through the pgStatFunctions to find functions that actually have
+ * counts, and try flushing it out to shared stats.
+ */
+ hash_seq_init(&scan, pgStatFunctions);
+ while ((bestat = (PgStat_BackendFunctionEntry *) hash_seq_search(&scan)) != NULL)
{
- PgStat_FunctionEntry *m_ent;
+ bool found;
+ PgStat_StatFuncEntry *shstat = NULL;
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&bestat->f_counts, &all_zeroes,
sizeof(PgStat_FunctionCounts)) == 0)
continue;
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
+ shstat = (PgStat_StatFuncEntry *)
+ dshash_find_extended(funchash, (void *) &(bestat->f_id),
+ true, nowait, true, &found);
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
+ /*
+ * We couldn't acquire lock on the required entry. Leave the local
+ * entry alone.
+ */
+ if (!shstat)
{
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
+ have_function_stats = true;
+ continue;
}
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
+ /* Initialize if it's new, or add to it. */
+ if (!found)
+ {
+ shstat->functionid = bestat->f_id;
+ shstat->f_numcalls = bestat->f_counts.f_numcalls;
+ shstat->f_total_time =
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_total_time);
+ shstat->f_self_time =
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_self_time);
+ }
+ else
+ {
+ shstat->f_numcalls += bestat->f_counts.f_numcalls;
+ shstat->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_total_time);
+ shstat->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(bestat->f_counts.f_self_time);
+ }
+ dshash_release_lock(funchash, shstat);
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
+ /* reset used counts */
+ MemSet(&bestat->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ }
- have_function_stats = false;
+ return have_function_stats;
}
/* ----------
* pgstat_vacuum_stat() -
*
- * Will tell the collector about objects he can get rid of.
+ * Remove objects we can get rid of.
* ----------
*/
void
pgstat_vacuum_stat(void)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
+ HTAB *oidtab;
+ dshash_seq_status dshstat;
PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
+ dshash_table_handle tables_handle;
+ dshash_table_handle functions_handle;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
return;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
/*
* Read pg_database and make a list of OIDs of all existing databases
*/
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ oidtab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
/*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
+ * Search the database hash table for dead databases and drop them from
+ * the hash.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+
+ dshash_seq_init(&dshstat, pgStatDBHash, false);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&dshstat)) != NULL)
{
Oid dbid = dbentry->databaseid;
@@ -1054,136 +955,48 @@ pgstat_vacuum_stat(void)
/* the DB entry for shared tables (with InvalidOid) is never dropped */
if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
+ hash_search(oidtab, (void *) &dbid, HASH_FIND, NULL) == NULL)
pgstat_drop_database(dbid);
}
/* Clean up */
- hash_destroy(htab);
+ dshash_seq_term(&dshstat);
+ hash_destroy(oidtab);
/*
* Lookup our own database entry; if not found, nothing more to do.
+ * MyDatabaseId cannot be removed or the hashes above are not changed, so
+ * we can release the lock just after.
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
+ dbentry = pgstat_get_db_entry(MyDatabaseId, false, false, false);
+ if (dbentry == NULL)
return;
+ tables_handle = dbentry->tables;
+ functions_handle = dbentry->functions;
+ dshash_release_lock(pgStatDBHash, dbentry);
/*
* Similarly to above, make a list of all known relations in this DB.
*/
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
-
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
+ oidtab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
/*
- * Send the rest
+ * Check for all tables listed in stats hash table if they still exist.
+ * Stats cache is useless here so directly search the shared hash.
*/
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
+ pgstat_remove_useless_entries(tables_handle, &dsh_tblparams, oidtab);
+ hash_destroy(oidtab);
/*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
+ * Repeat the above but we needn't bother in the common case where no
+ * function stats are being collected.
*/
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
+ oidtab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+ pgstat_remove_useless_entries(functions_handle, &dsh_funcparams,
+ oidtab);
+ hash_destroy(oidtab);
}
}
@@ -1212,7 +1025,7 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
hash_ctl.entrysize = sizeof(Oid);
hash_ctl.hcxt = CurrentMemoryContext;
htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
+ PGSTAT_TABLE_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
@@ -1239,65 +1052,96 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
}
-/* ----------
- * pgstat_drop_database() -
+/*
+ * pgstat_remove_useless_entries - Remove entries no loner exists from per
+ * table/function dshashes.
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Scan the dshash specified by dshhandle removing entries that are not in
+ * oidtab. oidtab is destroyed before returning.
*/
void
-pgstat_drop_database(Oid databaseid)
+pgstat_remove_useless_entries(const dshash_table_handle dshhandle,
+ const dshash_parameters *dshparams,
+ HTAB *oidtab)
{
- PgStat_MsgDropdb msg;
+ dshash_table *dshtable;
+ dshash_seq_status dshstat;
+ void *ent;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ dshtable = dshash_attach(area, dshparams, dshhandle, 0);
+ dshash_seq_init(&dshstat, dshtable, true);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* The first member of the entries must be Oid */
+ if (hash_search(oidtab, ent, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* Not there, so purge this entry */
+ dshash_delete_current(&dshstat);
+ }
+ /* clean up */
+ dshash_seq_term(&dshstat);
+ dshash_detach(dshtable);
+}
/* ----------
- * pgstat_drop_relation() -
+ * pgstat_drop_database() -
*
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
+ * Remove entry for the database that we just dropped.
*
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
+ * If some stats are flushed after this, this entry will be re-created but we
+ * will still clean the dead DB eventually via future invocations of
+ * pgstat_vacuum_stat().
* ----------
*/
-#ifdef NOT_USED
void
-pgstat_drop_relation(Oid relid)
+pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgTabpurge msg;
- int len;
+ PgStat_StatDBEntry *dbentry;
+
+ Assert(OidIsValid(databaseid));
+
+ if (!IsUnderPostmaster || !pgStatDBHash)
+ return;
+
+ /*
+ * Lookup the database, removal needs exclusive lock.
+ */
+ dbentry = pgstat_get_db_entry(databaseid, true, false, false);
- if (pgStatSock == PGINVALID_SOCKET)
+ if (dbentry == NULL)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
+ /* found, remove it */
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
+ /* Remove table/function stats dshash first. */
+ if (dbentry->tables != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-}
-#endif /* NOT_USED */
+ dshash_destroy(tbl);
+ }
+ if (dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ dshash_table *tbl =
+ dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+
+ dshash_destroy(tbl);
+ }
+
+ dshash_delete_entry(pgStatDBHash, (void *) dbentry);
+}
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1306,20 +1150,30 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStat_StatDBEntry *dbentry;
+
+ if (!pgStatDBHash)
+ return;
- if (pgStatSock == PGINVALID_SOCKET)
+ /*
+ * Lookup the database in the hash table. Nothing to do if not there.
+ * This function works on the dbentry, so we cannot release it earlier.
+ */
+ dbentry = pgstat_get_db_entry(MyDatabaseId, false, false, false);
+ if (!dbentry)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /* Reset database-level stats. */
+ reset_dbentry_counters(dbentry);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
+
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1328,29 +1182,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ /* Reset the bgwriter statistics for the cluster. */
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1359,17 +1221,39 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
+ dshash_table *t;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ dbentry = pgstat_get_db_entry(MyDatabaseId, false, false, false);
+ Assert(dbentry);
+
+ /* Set the reset timestamp for the whole database */
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&dbentry->lock);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /*
+ * MyDatabaseId cannot be removed or the hashes above are not changed, so
+ * we can release the lock right now.
+ */
+ dshash_release_lock(pgStatDBHash, dbentry);
+
+ /* Remove object if it exists, ignore if not */
+ if (type == RESET_TABLE)
+ {
+ t = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_delete_key(t, (void *) &objoid);
+ dshash_detach(t);
+ }
- pgstat_send(&msg, sizeof(msg));
+ if (type == RESET_FUNCTION && dbentry->functions != DSM_HANDLE_INVALID)
+ {
+ t = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_delete_key(t, (void *) &objoid);
+ dshash_detach(t);
+ }
}
/* ----------
@@ -1383,48 +1267,87 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ ts = GetCurrentTimestamp();
+
+ /*
+ * Store the last autovacuum time in the database's hash table entry.
+ */
+ dbentry = pgstat_get_db_entry(dboid, false, false, true);
+ Assert(!dbentry);
+
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->last_autovac_time = ts;
+ LWLockRelease(&dbentry->lock);
- pgstat_send(&msg, sizeof(msg));
+ dshash_release_lock(pgStatDBHash, dbentry);
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table_handle table_handle;
+ dshash_table *table;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hash table entry. The dshash table cannot
+ * be destroyed meanwhile, so release the dbent right now.
+ */
+ dbentry = pgstat_get_db_entry(dboid, false, false, true);
+ Assert(dbentry);
+ table_handle = dbentry->tables;
+ dshash_release_lock(pgStatDBHash, dbentry);
+
+ ts = GetCurrentTimestamp();
+
+ table = dshash_attach(area, &dsh_tblparams, table_handle, 0);
+ tabentry = pgstat_get_tab_entry(table, tableoid, true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->vacuum_count++;
+ }
+ dshash_release_lock(table, tabentry);
+ dshash_detach(table);
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1435,9 +1358,14 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ Oid dboid;
+ PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabentry;
+ dshash_table_handle table_handle;
+ dshash_table *table;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1445,10 +1373,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1466,84 +1394,125 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ dboid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Store the data in the table's hash table entry. The dshash table cannot
+ * be destroyed meanwhile, so release the dbent right now.
+ */
+ dbentry = pgstat_get_db_entry(dboid, false, false, true);
+ Assert(dbentry);
+ table_handle = dbentry->tables;
+ dshash_release_lock(pgStatDBHash, dbentry);
+
+ table = dshash_attach(area, &dsh_tblparams, table_handle, 0);
+ tabentry = pgstat_get_tab_entry(table, RelationGetRelid(rel), true);
+
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+
+ dshash_release_lock(table, tabentry);
+ dshash_detach(table);
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
-}
-
-/* --------
- * pgstat_report_deadlock() -
- *
- * Tell the collector about a deadlock detected.
- * --------
- */
-void
-pgstat_report_deadlock(void)
-{
- PgStat_MsgDeadlock msg;
+ dbentry = pgstat_get_db_entry(MyDatabaseId, false, false, false);
+ Assert(dbentry);
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbentry->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbentry->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbentry->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbentry->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbentry->counts.n_conflict_startup_deadlock++;
+ break;
+ }
+ LWLockRelease(&dbentry->lock);
+ dshash_release_lock(pgStatDBHash, dbentry);
}
-
/* --------
- * pgstat_report_checksum_failures_in_db() -
+ * pgstat_report_deadlock() -
*
- * Tell the collector about one or more checksum failures.
+ * Report a deadlock detected.
* --------
*/
void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
+pgstat_report_deadlock(void)
{
- PgStat_MsgChecksumFailure msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
+ dbentry = pgstat_get_db_entry(MyDatabaseId, false, false, false);
+ Assert(dbentry);
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->counts.n_deadlocks++;
+ LWLockRelease(&dbentry->lock);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
}
+
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
@@ -1555,60 +1524,153 @@ pgstat_report_checksum_failure(void)
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ dbentry = pgstat_get_db_entry(MyDatabaseId, false, false, false);
+ Assert(dbentry);
+
+ if (filesize > 0) /* Is there a case where filesize is really 0? */
+ {
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbentry->counts.n_temp_files++;
+ LWLockRelease(&dbentry->lock);
+ }
+
+ dshash_release_lock(pgStatDBHash, dbentry);
}
-/* ----------
- * pgstat_ping() -
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Send some junk data to the collector to increase traffic.
- * ----------
+ * Reports about one or more checksum failures.
+ * --------
*/
void
-pgstat_ping(void)
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgDummy msg;
+ PgStat_StatDBEntry *dbentry;
+ ChecksumFailureEnt *failent = NULL;
+
+ /* return if we are not active */
+ if (!area)
+ return;
+
+ /* add accumulated count to the parameter */
+ if (checksum_failures != NULL)
+ {
+ failent = hash_search(checksum_failures, &dboid, HASH_FIND, NULL);
+ if (failent)
+ failurecount += failent->count;
+ }
+
+ if (failurecount == 0)
+ return;
+
+ dbentry = pgstat_get_db_entry(MyDatabaseId, false, true, false);
+
+ if (!dbentry)
+ {
+ /* failed to acquire shared entry, store the number locally */
+ if (!failent)
+ {
+ bool found;
+
+ if (!checksum_failures)
+ {
+ HASHCTL ctl;
+
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(ChecksumFailureEnt);
+ checksum_failures =
+ hash_create("pgstat checksum failure count hash",
+ 32, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+
+ failent = hash_search(checksum_failures, &dboid, HASH_ENTER,
+ &found);
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!found)
+ nchecksum_failures++;
+ }
+
+ failent->count = failurecount;
return;
+ }
+
+ /* We have a chance to flush immediately */
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->counts.n_checksum_failures += failurecount;
+ LWLockRelease(&dbentry->lock);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dshash_release_lock(pgStatDBHash, dbentry);
+
+ if (checksum_failures)
+ {
+ /* Remove the entry and the hash if it gets empty */
+ hash_search(checksum_failures, &dboid, HASH_REMOVE, NULL);
+
+ if (failent != NULL && --nchecksum_failures < 1)
+ {
+ hash_destroy(checksum_failures);
+ checksum_failures = NULL;
+ }
+ }
}
-/* ----------
- * pgstat_send_inquiry() -
+/*
+ * flush checkpoint failure count for all databases
*
- * Notify collector that we need fresh data.
- * ----------
+ * Returns true if some entries are left unflushed.
*/
static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
+pgstat_flush_checksum_failure(bool nowait)
{
- PgStat_MsgInquiry msg;
+ HASH_SEQ_STATUS stat;
+ ChecksumFailureEnt *ent;
+ PgStat_StatDBEntry *dbentry;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
+ if (checksum_failures == NULL)
+ return;
+
+ hash_seq_init(&stat, checksum_failures);
+ while ((ent = (ChecksumFailureEnt *) hash_seq_search(&stat)) != NULL)
+ {
+ dbentry = pgstat_get_db_entry(ent->dboid, false, nowait, true);
+ if (dbentry)
+ {
+ /* update database stats */
+ LWLockAcquire(&dbentry->lock, LW_EXCLUSIVE);
+ dbentry->counts.n_checksum_failures += ent->count;
+ LWLockRelease(&dbentry->lock);
+
+ hash_search(checksum_failures, &ent->dboid, HASH_REMOVE, NULL);
+
+ dshash_release_lock(pgStatDBHash, dbentry);
+ nchecksum_failures--;
+ }
+ }
+
+ /* The hash is empty, destroy it. */
+ if (nchecksum_failures < 1)
+ {
+ hash_destroy(checksum_failures);
+ checksum_failures = NULL;
+ }
+ return;
+}
/*
* Initialize function call usage data.
@@ -1739,8 +1801,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1760,7 +1821,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1779,85 +1841,45 @@ pgstat_initstats(Relation rel)
rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
+
/*
* get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
*/
static PgStat_TableStatus *
get_tabstat_entry(Oid rel_id, bool isshared)
{
- TabStatHashEntry *hash_entry;
PgStat_TableStatus *entry;
- TabStatusArray *tsa;
bool found;
/*
* Create hash table if we don't have it already.
*/
- if (pgStatTabHash == NULL)
+ if (pgStatTables == NULL)
{
HASHCTL ctl;
- memset(&ctl, 0, sizeof(ctl));
+ MemSet(&ctl, 0, sizeof(ctl));
ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
+ ctl.entrysize = sizeof(PgStat_TableStatus);
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ pgStatTables = hash_create("Table stat entries",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
}
/*
* Find an entry or create a new one.
*/
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
+ entry = hash_search(pgStatTables, &rel_id, HASH_ENTER, &found);
if (!found)
{
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
- }
-
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
-
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
+ entry->t_shared = isshared;
+ entry->trans = NULL;
+ MemSet(&entry->t_counts, 0, sizeof(PgStat_TableCounts));
}
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
-
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
+ have_table_stats = true;
return entry;
}
@@ -1866,26 +1888,16 @@ get_tabstat_entry(Oid rel_id, bool isshared)
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
* If no entry, return NULL, don't create a new one
- *
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
-
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
-
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
+ /* If hash table doesn't exist, there are no entries at all */
+ if (!pgStatTables)
return NULL;
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return (PgStat_TableStatus *) hash_search(pgStatTables,
+ &rel_id, HASH_FIND, NULL);
}
/*
@@ -2315,9 +2327,9 @@ AtPrepare_PgStat(void)
* Clean up after successful PREPARE.
*
* All we need do here is unlink the transaction stats state from the
- * nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
+ * nontransactional state. The nontransactional action counts will be reported
+ * immediately, while the effects on live and dead tuple counts are preserved
+ * in the 2PC state file.
*
* Note: AtEOXact_PgStat is not called during PREPARE.
*/
@@ -2415,91 +2427,248 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
}
-/* ----------
- * pgstat_fetch_stat_dbentry() -
+/*
+ * snapshot_statentry() - Common routine for functions
+ * pgstat_fetch_stat_*entry()
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
- * ----------
+ * Returns the pointer to a snapshot of a shared entry for the key or NULL if
+ * not found. Returned snapshots are stable during the current transaction or
+ * until pgstat_clear_snapshot() is called.
+ *
+ * Created snapshots are stored in HTAB *snapshot_hash. If not created yet, it
+ * is created using snapshot_hash_name, snapshot_hash_entsize.
+ *
+ * *table_hash points to dshash_table. If not yet attached, it is attached
+ * using table_hash_params and table_hash_handle.
*/
-PgStat_StatDBEntry *
-pgstat_fetch_stat_dbentry(Oid dbid)
+static void *
+snapshot_statentry(const Oid key, const char *snapshot_hash_name,
+ const int snapshot_hash_entsize,
+ const dshash_table_handle table_hash_handle,
+ const dshash_parameters *table_hash_params,
+ HTAB **snapshot_hash, dshash_table **table_hash)
{
+ PgStat_snapshot *lentry = NULL;
+ size_t table_hash_keysize = table_hash_params->key_size;
+ size_t table_hash_entrysize = table_hash_params->entry_size;
+ bool found;
+
+ /*
+ * We don't want so frequent update of stats snapshot. Keep it at least
+ * for PGSTAT_STAT_MIN_INTERVAL ms. Not postpone but just ignore the cue.
+ */
+ if (clear_snapshot)
+ {
+ clear_snapshot = false;
+
+ if (pgStatSnapshotContext &&
+ snapshot_globalStats.stats_timestamp <
+ GetCurrentStatementStartTimestamp() -
+ PGSTAT_STAT_MIN_INTERVAL * 1000)
+ {
+ MemoryContextReset(pgStatSnapshotContext);
+
+ /* Reset variables */
+ global_snapshot_is_valid = false;
+ pgStatSnapshotContext = NULL;
+ pgStatLocalHash = NULL;
+
+ pgstat_setup_memcxt();
+ *snapshot_hash = NULL;
+ }
+ }
+
/*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
+ * Create new hash, with rather arbitrary initial number of entries since
+ * we don't know how this hash will grow.
*/
- backend_read_statsfile();
+ if (!*snapshot_hash)
+ {
+ HASHCTL ctl;
+
+ /*
+ * Create the hash in the stats context
+ *
+ * The entry is prepended by common header part represented by
+ * PgStat_snapshot.
+ */
+
+ ctl.keysize = table_hash_keysize;
+ ctl.entrysize =
+ offsetof(PgStat_snapshot, body) + snapshot_hash_entsize;
+ ctl.hcxt = pgStatSnapshotContext;
+ *snapshot_hash = hash_create(snapshot_hash_name, 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ lentry = hash_search(*snapshot_hash, &key, HASH_ENTER, &found);
/*
- * Lookup the requested database; return NULL if not found
+ * Refer shared hash if not found in the local hash. We return up-to-date
+ * entries outside a transaction so do the same even if the snapshot is
+ * found.
*/
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ if (!found || !IsTransactionState())
+ {
+ void *sentry;
+
+ /* attach shared hash if not given, leave it alone for later use */
+ if (!*table_hash)
+ {
+ MemoryContext oldcxt;
+
+ Assert(table_hash_handle != DSM_HANDLE_INVALID);
+ oldcxt = MemoryContextSwitchTo(pgStatSnapshotContext);
+ *table_hash =
+ dshash_attach(area, table_hash_params, table_hash_handle, NULL);
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ sentry = dshash_find(*table_hash, &key, false);
+
+ if (sentry)
+ {
+ /*
+ * In transaction state, it is obvious that we should create local
+ * cache entries for consistency. If we are not, we return an
+ * up-to-date entry. Having said that, we need a local copy since
+ * dshash entry must be released immediately. We share the same
+ * local hash entry for the purpose.
+ */
+ memcpy(&lentry->body, sentry, table_hash_entrysize);
+ dshash_release_lock(*table_hash, sentry);
+
+ /* then zero out the local additional space if any */
+ if (table_hash_entrysize < snapshot_hash_entsize)
+ MemSet((char *) &lentry->body + table_hash_entrysize, 0,
+ snapshot_hash_entsize - table_hash_entrysize);
+ }
+
+ lentry->negative = !sentry;
+ }
+
+ if (lentry->negative)
+ return NULL;
+
+ return &lentry->body;
}
+/* ----------
+ * pgstat_fetch_stat_dbentry_snapshot() -
+ *
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatDBEntry *
+pgstat_fetch_stat_dbentry(Oid dbid)
+{
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *)
+ snapshot_statentry(dbid,
+ "local database stats", /* snapshot hash name */
+ sizeof(PgStat_StatDBEntry), /* snapshot ent size */
+ DSM_HANDLE_INVALID, /* dshash handle */
+ &dsh_dbparams, /* dshash params */
+ &pgStatLocalHash, /* snapshot hash */
+ &pgStatDBHash); /* shared hash */
+}
+
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* Lookup our database, then look in its table hash table. */
+ dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
+ if (dbentry == NULL)
+ return NULL;
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(dbentry, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ dbentry = pgstat_fetch_stat_dbentry(InvalidOid);
+ if (dbentry == NULL)
+ return NULL;
+
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(dbentry, relid);
+ if (tabentry != NULL)
+ return tabentry;
return NULL;
}
+/* ----------
+ * pgstat_fetch_stat_tabentry_snapshot() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_snapshot(PgStat_StatDBEntry *dbent, Oid reloid)
+{
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(reloid,
+ "table stats snapshot", /* snapshot hash name */
+ sizeof(PgStat_StatTabEntry), /* snapshot ent size */
+ dbent->tables, /* dshash handle */
+ &dsh_tblparams, /* dshash params */
+ &dbent->snapshot_tables, /* snapshot hash */
+ &dbent->dshash_tables); /* shared hash */
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Sets index counters to specified
+ * place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
+}
+
+
/* ----------
* pgstat_fetch_stat_funcentry() -
*
@@ -2513,49 +2682,103 @@ pgstat_fetch_stat_funcentry(Oid func_id)
PgStat_StatDBEntry *dbentry;
PgStat_StatFuncEntry *funcentry = NULL;
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
+ /* Lookup our database, then find the requested function */
dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
+ if (dbentry == NULL)
+ return NULL;
+
+ funcentry = pgstat_fetch_stat_funcentry_snapshot(dbentry, func_id);
return funcentry;
}
-
/* ----------
- * pgstat_fetch_stat_beentry() -
- *
- * Support function for the SQL-callable pgstat* functions. Returns
- * our local copy of the current-activity entry for one backend.
+ * pgstat_fetch_stat_funcentry_snapshot() -
*
- * NB: caller is responsible for a check if the user is permitted to see
- * this info (especially the querystring).
- * ----------
+ * Find function stats entry on backends in dbent. The returned entry is
+ * cached until transaction end or pgstat_clear_snapshot() is called.
*/
-PgBackendStatus *
-pgstat_fetch_stat_beentry(int beid)
+static PgStat_StatFuncEntry *
+pgstat_fetch_stat_funcentry_snapshot(PgStat_StatDBEntry *dbent, Oid funcid)
{
- pgstat_read_current_status();
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- if (beid < 1 || beid > localNumBackends)
+ if (dbent->functions == DSM_HANDLE_INVALID)
return NULL;
- return &localBackendStatusTable[beid - 1].backendStatus;
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(funcid,
+ "function stats snapshot", /* snapshot hash name */
+ sizeof(PgStat_StatFuncEntry), /* snapshot ent size */
+ dbent->functions, /* dshash handle */
+ &dsh_funcparams, /* dshash params */
+ &dbent->snapshot_functions, /* snapshot hash */
+ &dbent->dshash_functions); /* shared hash */
}
-
-/* ----------
- * pgstat_fetch_stat_local_beentry() -
- *
- * Like pgstat_fetch_stat_beentry() but with locally computed additions (like
- * xid and xmin values of the backend)
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+
+ pgstat_attach_shared_stats();
+
+ /* Nothing to do if already done */
+ if (global_snapshot_is_valid)
+ return;
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ memcpy(&snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ LWLockRelease(StatsLock);
+
+ global_snapshot_is_valid = true;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
+}
+
+/* ----------
+ * pgstat_fetch_stat_beentry() -
+ *
+ * Support function for the SQL-callable pgstat* functions. Returns
+ * our local copy of the current-activity entry for one backend.
+ *
+ * NB: caller is responsible for a check if the user is permitted to see
+ * this info (especially the querystring).
+ * ----------
+ */
+PgBackendStatus *
+pgstat_fetch_stat_beentry(int beid)
+{
+ pgstat_read_current_status();
+
+ if (beid < 1 || beid > localNumBackends)
+ return NULL;
+
+ return &localBackendStatusTable[beid - 1].backendStatus;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_local_beentry() -
+ *
+ * Like pgstat_fetch_stat_beentry() but with locally computed additions (like
+ * xid and xmin values of the backend)
*
* NB: caller is responsible for a check if the user is permitted to see
* this info (especially the querystring).
@@ -2599,9 +2822,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return &snapshot_archiverStats;
}
@@ -2616,9 +2840,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return &snapshot_globalStats;
}
@@ -2832,8 +3057,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3009,12 +3234,16 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+
+ /* attach shared database stats area */
+ pgstat_attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3027,7 +3256,7 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
@@ -3044,6 +3273,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ pgstat_detach_shared_stats(true);
}
@@ -3304,7 +3535,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3599,9 +3831,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_ALL:
event_name = "RecoveryWalAll";
break;
@@ -4221,94 +4450,71 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
- *
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
+ * pgstat_report_archiver() -
*
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Report archiver statistics
* ----------
*/
void
-pgstat_send_archiver(const char *xlog, bool failed)
+pgstat_report_archiver(const char *xlog, bool failed)
{
- PgStat_MsgArchiver msg;
+ TimestampTz now = GetCurrentTimestamp();
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ if (failed)
+ {
+ /* Failed archival attempt */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ /* Successful archival operation */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *s = &BgWriterStats;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty message.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += s->timed_checkpoints;
+ shared_globalStats->requested_checkpoints += s->requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += s->checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += s->checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += s->buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += s->buf_written_clean;
+ shared_globalStats->maxwritten_clean += s->maxwritten_clean;
+ shared_globalStats->buf_written_backend += s->buf_written_backend;
+ shared_globalStats->buf_fsync_backend += s->buf_fsync_backend;
+ shared_globalStats->buf_alloc += s->buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4317,422 +4523,162 @@ pgstat_send_bgwriter(void)
}
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+static void
+init_dbentry(PgStat_StatDBEntry *dbentry)
{
- int len;
- PgStat_Msg msg;
- int wr;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
+ dshash_table *tabhash;
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
+ LWLockInitialize(&dbentry->lock, LWTRANCHE_STATS);
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ /* initialize the new shared entry */
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
+ dbentry->functions = DSM_HANDLE_INVALID;
- exit(0);
+ /* dbentry always has the table hash */
+ tabhash = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->tables = dshash_get_hash_table_handle(tabhash);
+ dshash_detach(tabhash);
}
+
/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
+ * Create the filename for a DB stat file; filename is output parameter points
+ * to a character buffer of length len.
*/
static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
+ int printed;
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
}
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
+
+static void
+reset_tabcount(PgStat_StatTabEntry *ent)
{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
+ ent->numscans = 0;
+ ent->tuples_returned = 0;
+ ent->tuples_fetched = 0;
+ ent->tuples_inserted = 0;
+ ent->tuples_updated = 0;
+ ent->tuples_deleted = 0;
+ ent->tuples_hot_updated = 0;
+ ent->n_live_tuples = 0;
+ ent->n_dead_tuples = 0;
+ ent->changes_since_analyze = 0;
+ ent->blocks_fetched = 0;
+ ent->blocks_hit = 0;
+ ent->vacuum_count = 0;
+ ent->autovac_vacuum_count = 0;
+ ent->analyze_count = 0;
+ ent->autovac_analyze_count = 0;
+
+ ent->vacuum_timestamp = 0;
+ ent->autovac_vacuum_timestamp = 0;
+ ent->analyze_timestamp = 0;
+ ent->autovac_analyze_timestamp = 0;
+}
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
- if (!create && !found)
- return NULL;
+static void
+reset_dbcount(PgStat_StatDBEntry *ent)
+{
+ TimestampTz ts = GetCurrentTimestamp();
+
+ LWLockAcquire(&ent->lock, LW_EXCLUSIVE);
+
+ ent->counts.n_tuples_returned = 0;
+ ent->counts.n_tuples_fetched = 0;
+ ent->counts.n_tuples_inserted = 0;
+ ent->counts.n_tuples_updated = 0;
+ ent->counts.n_tuples_deleted = 0;
+ ent->counts.n_blocks_fetched = 0;
+ ent->counts.n_blocks_hit = 0;
+ ent->counts.n_xact_commit = 0;
+ ent->counts.n_xact_rollback = 0;
+ ent->counts.n_block_read_time = 0;
+ ent->counts.n_block_write_time = 0;
+ ent->stat_reset_timestamp = ts;
+
+ LWLockRelease(&ent->lock);
+}
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
- return result;
+static void
+reset_funccount(PgStat_StatFuncEntry *ent)
+{
+ ent->f_numcalls = 0;
+ ent->f_total_time = 0;
+ ent->f_self_time = 0;
}
/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+ * Subroutine to clear stats in a database entry
+ *
+ * Reset all counters in the dbentry.
*/
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
+static void
+reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
+ dshash_table *tbl;
+ dshash_seq_status dshstat;
+ PgStat_StatTabEntry *tabent;
+ PgStat_StatFuncEntry *funcent;
+
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&dshstat, tbl, true);
+ while ((tabent = dshash_seq_next(&dshstat)) != NULL)
+ reset_tabcount(tabent);
+ dshash_seq_term(&dshstat);
+ dshash_detach(tbl);
+
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->functions, 0);
+ dshash_seq_init(&dshstat, tbl, true);
+ while ((funcent = dshash_seq_next(&dshstat)) != NULL)
+ reset_funccount(funcent);
+ dshash_seq_term(&dshstat);
+ dshash_detach(tbl);
}
- return result;
+ reset_dbcount(dbentry);
}
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
+ dshash_seq_status hstat;
PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
+
elog(DEBUG2, "writing stats file \"%s\"", statfile);
/*
@@ -4751,7 +4697,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4763,32 +4709,29 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_init(&hstat, pgStatDBHash, false);
+ while ((dbentry = (PgStat_StatDBEntry *) dshash_seq_next(&hstat)) != NULL)
{
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_pgStatDBHashfile(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
@@ -4798,6 +4741,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ dshash_seq_term(&hstat);
/*
* No more output to be done. Close the temp file and replace the old
@@ -4831,53 +4775,19 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
}
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
/* ----------
- * pgstat_write_db_statsfile() -
+ * pgstat_write_pgStatDBHashfile() -
* Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_pgStatDBHashfile(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
+ dshash_seq_status tstat;
+ dshash_seq_status fstat;
PgStat_StatTabEntry *tabentry;
PgStat_StatFuncEntry *funcentry;
FILE *fpout;
@@ -4886,9 +4796,10 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
int rc;
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
+ dshash_table *tbl;
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -4915,23 +4826,34 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ Assert(dbentry->tables != DSM_HANDLE_INVALID);
+
+ tbl = dshash_attach(area, &dsh_tblparams, dbentry->tables, 0);
+ dshash_seq_init(&tstat, tbl, false);
+ while ((tabentry = (PgStat_StatTabEntry *) dshash_seq_next(&tstat)) != NULL)
{
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ dshash_seq_term(&tstat);
+ dshash_detach(tbl);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ if (dbentry->functions != DSM_HANDLE_INVALID)
{
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ tbl = dshash_attach(area, &dsh_funcparams, dbentry->functions, 0);
+ dshash_seq_init(&fstat, tbl, false);
+ while ((funcentry = (PgStat_StatFuncEntry *) dshash_seq_next(&fstat)) != NULL)
+ {
+ fputc('F', fpout);
+ rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+ }
+ dshash_seq_term(&fstat);
+ dshash_detach(tbl);
}
/*
@@ -4966,94 +4888,56 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
}
+
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * Reads in existing activity statistics files into the shared stats hash.
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5062,7 +4946,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5070,38 +4954,30 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5115,7 +4991,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
fpin) != offsetof(PgStat_StatDBEntry, tables))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5124,76 +5000,36 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+ dbentry = (PgStat_StatDBEntry *)
+ dshash_find_or_insert(pgStatDBHash, (void *) &dbbuf.databaseid,
+ &found);
+
+ /* don't allow duplicate dbentries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(pgStatDBHash, dbentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
+ init_dbentry(dbentry);
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, tables));
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
+ Assert(dbentry->tables != DSM_HANDLE_INVALID);
+ /* Read the data from the database-specific file. */
+ pgstat_read_pgStatDBHashfile(dbentry);
+ dshash_release_lock(pgStatDBHash, dbentry);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5203,59 +5039,49 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
/* ----------
- * pgstat_read_db_statsfile() -
- *
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
+ * pgstat_read_pgStatDBHashfile() -
*
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_pgStatDBHashfile(PgStat_StatDBEntry *dbentry)
{
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
FILE *fpin;
int32 format_id;
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5268,14 +5094,17 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
+ /* Create table stats hash */
+ Assert(dbentry->tables != DSM_HANDLE_INVALID);
+
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing statistics file. Read it and put all the hash
+ * table entries into place.
*/
for (;;)
{
@@ -5288,31 +5117,32 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
+ if (!tabhash)
+ tabhash = dshash_attach(area, &dsh_tblparams,
+ dbentry->tables, 0);
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ tabentry = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(tabhash,
+ (void *) &tabbuf.tableid, &found);
+ /* don't allow duplicate entries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(tabhash, tabentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
memcpy(tabentry, &tabbuf, sizeof(tabbuf));
+ dshash_release_lock(tabhash, tabentry);
break;
/*
@@ -5322,31 +5152,34 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if function data not wanted.
- */
if (funchash == NULL)
- break;
+ {
+ funchash = dshash_create(area, &dsh_tblparams, 0);
+ dbentry->functions =
+ dshash_get_hash_table_handle(funchash);
+ }
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ funcentry = (PgStat_StatFuncEntry *)
+ dshash_find_or_insert(funchash,
+ (void *) &funcbuf.functionid, &found);
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ dshash_release_lock(funchash, funcentry);
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
memcpy(funcentry, &funcbuf, sizeof(funcbuf));
+ dshash_release_lock(funchash, funcentry);
break;
/*
@@ -5356,7 +5189,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5364,292 +5197,38 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
-done:
FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
/* ----------
* pgstat_setup_memcxt() -
*
- * Create pgStatLocalContext, if not already done.
+ * Create pgStatLocalContext and pgStatSnapshotContext, if not already done.
* ----------
*/
static void
pgstat_setup_memcxt(void)
{
if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+
+ if (!pgStatSnapshotContext)
+ pgStatSnapshotContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Database statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
}
@@ -5668,739 +5247,185 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
+ }
+ if (pgStatSnapshotContext)
+ clear_snapshot = true;
+}
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
+static bool
+pgstat_update_tabentry(dshash_table *tabhash, PgStat_TableStatus *stat,
+ bool nowait)
{
- PgStat_StatDBEntry *dbentry;
+ PgStat_StatTabEntry *tabent;
+ bool found;
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
+ if (tabhash == NULL)
+ return false;
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
+ tabent = (PgStat_StatTabEntry *)
+ dshash_find_extended(tabhash, (void *) &(stat->t_id),
+ true, nowait, true, &found);
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
+ /* failed to acquire lock */
+ if (tabent == NULL)
+ return false;
+
+ if (!found)
{
/*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
+ * If it's a new table entry, initialize counters to the values we
+ * just got.
*/
+ tabent->numscans = stat->t_counts.t_numscans;
+ tabent->tuples_returned = stat->t_counts.t_tuples_returned;
+ tabent->tuples_fetched = stat->t_counts.t_tuples_fetched;
+ tabent->tuples_inserted = stat->t_counts.t_tuples_inserted;
+ tabent->tuples_updated = stat->t_counts.t_tuples_updated;
+ tabent->tuples_deleted = stat->t_counts.t_tuples_deleted;
+ tabent->tuples_hot_updated = stat->t_counts.t_tuples_hot_updated;
+ tabent->n_live_tuples = stat->t_counts.t_delta_live_tuples;
+ tabent->n_dead_tuples = stat->t_counts.t_delta_dead_tuples;
+ tabent->changes_since_analyze = stat->t_counts.t_changed_tuples;
+ tabent->blocks_fetched = stat->t_counts.t_blocks_fetched;
+ tabent->blocks_hit = stat->t_counts.t_blocks_hit;
+
+ tabent->vacuum_timestamp = 0;
+ tabent->vacuum_count = 0;
+ tabent->autovac_vacuum_timestamp = 0;
+ tabent->autovac_vacuum_count = 0;
+ tabent->analyze_timestamp = 0;
+ tabent->analyze_count = 0;
+ tabent->autovac_analyze_timestamp = 0;
+ tabent->autovac_analyze_count = 0;
}
- else if (msg->clock_time < dbentry->stats_timestamp)
+ else
{
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
+ /*
+ * Otherwise add the values to the existing entry.
+ */
+ tabent->numscans += stat->t_counts.t_numscans;
+ tabent->tuples_returned += stat->t_counts.t_tuples_returned;
+ tabent->tuples_fetched += stat->t_counts.t_tuples_fetched;
+ tabent->tuples_inserted += stat->t_counts.t_tuples_inserted;
+ tabent->tuples_updated += stat->t_counts.t_tuples_updated;
+ tabent->tuples_deleted += stat->t_counts.t_tuples_deleted;
+ tabent->tuples_hot_updated += stat->t_counts.t_tuples_hot_updated;
+ /* If table was truncated, first reset the live/dead counters */
+ if (stat->t_counts.t_truncated)
{
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
+ tabent->n_live_tuples = 0;
+ tabent->n_dead_tuples = 0;
}
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
+ tabent->n_live_tuples += stat->t_counts.t_delta_live_tuples;
+ tabent->n_dead_tuples += stat->t_counts.t_delta_dead_tuples;
+ tabent->changes_since_analyze += stat->t_counts.t_changed_tuples;
+ tabent->blocks_fetched += stat->t_counts.t_blocks_fetched;
+ tabent->blocks_hit += stat->t_counts.t_blocks_hit;
}
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ tabent->n_live_tuples = Max(tabent->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ tabent->n_dead_tuples = Max(tabent->n_dead_tuples, 0);
+
+ dshash_release_lock(tabhash, tabent);
+
+ return true;
}
-/* ----------
- * pgstat_recv_tabstat() -
+/*
+ * Lookup shared stats hash table for the specified database. Returns NULL
+ * when PGSTAT_NOWAIT and required lock cannot be acquired.
*
- * Count what the backend has done.
- * ----------
*/
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
+static PgStat_StatDBEntry *
+pgstat_get_db_entry(Oid databaseid, bool exclusive, bool nowait, bool create)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
+ PgStat_StatDBEntry *result;
+ bool found = true;
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
+ if (!IsUnderPostmaster || !pgStatDBHash)
+ return NULL;
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
+ /* Lookup or create the hash table entry for this database */
+ result = (PgStat_StatDBEntry *)
+ dshash_find_extended(pgStatDBHash, &databaseid,
+ exclusive, nowait, create, &found);
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
+ if (result == NULL)
+ return NULL;
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
+ if (create && !found)
+ {
+ Assert(create);
/*
- * Add per-table stats to the per-database entry, too.
+ * Initialize the new entry. This creates empty hash tables hash,
+ * too.
*/
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
+ init_dbentry(result);
}
-}
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
+ return result;
}
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
+/*
+ * Lookup the hash table entry for the specified table. Returned entry is
+ * exclusive locked.
+ * If no hash table entry exists, creates it, if create is true.
+ * Else, returns NULL.
*/
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
+static PgStat_StatTabEntry *
+pgstat_get_tab_entry(dshash_table *table, Oid tableoid, bool create)
{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
+ PgStat_StatTabEntry *result;
bool found;
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
+ /* Lookup or create the hash table entry for this table */
+ if (create)
+ result = (PgStat_StatTabEntry *)
+ dshash_find_or_insert(table, &tableoid, &found);
+ else
+ result = (PgStat_StatTabEntry *) dshash_find(table, &tableoid, false);
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
+ if (!create && !found)
+ return NULL;
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
+ /* If not found, initialize the new one. */
+ if (!found)
{
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
+ result->numscans = 0;
+ result->tuples_returned = 0;
+ result->tuples_fetched = 0;
+ result->tuples_inserted = 0;
+ result->tuples_updated = 0;
+ result->tuples_deleted = 0;
+ result->tuples_hot_updated = 0;
+ result->n_live_tuples = 0;
+ result->n_dead_tuples = 0;
+ result->changes_since_analyze = 0;
+ result->blocks_fetched = 0;
+ result->blocks_hit = 0;
+ result->vacuum_timestamp = 0;
+ result->vacuum_count = 0;
+ result->autovac_vacuum_timestamp = 0;
+ result->autovac_vacuum_count = 0;
+ result->analyze_timestamp = 0;
+ result->analyze_count = 0;
+ result->autovac_analyze_timestamp = 0;
+ result->autovac_analyze_count = 0;
}
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
+ return result;
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index fab4a9dd51..d418fe3bd0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -255,7 +255,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -503,7 +502,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1326,12 +1324,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1780,11 +1772,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2694,8 +2681,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3058,8 +3043,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3126,13 +3109,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3205,22 +3181,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3681,22 +3641,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3892,8 +3836,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3928,8 +3870,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4130,8 +4071,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5109,18 +5048,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5239,12 +5166,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6139,7 +6060,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6195,8 +6115,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6431,7 +6349,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 5880054245..04445c4c76 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2000,7 +2000,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ BgWriterStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2110,7 +2110,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2300,7 +2300,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2308,7 +2308,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 427b0d59cd..58a442f482 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 4c14e51c67..f61fd3e8ad 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -523,6 +523,7 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
+ LWLockRegisterTranche(LWTRANCHE_STATS, "activity_statistics");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 00c77b66c7..e2998f965e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3189,6 +3189,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3763,6 +3769,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4201,6 +4208,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4213,8 +4222,13 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4249,7 +4263,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4257,6 +4271,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index cea01534a5..a1304dc3ce 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -33,9 +33,6 @@
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1244,7 +1241,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1260,7 +1257,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1276,7 +1273,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1292,7 +1289,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1308,7 +1305,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1324,7 +1321,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1340,7 +1337,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1356,7 +1353,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1372,7 +1369,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1405,7 +1402,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1421,7 +1418,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1436,7 +1433,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1451,7 +1448,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1466,7 +1463,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1481,7 +1478,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1496,7 +1493,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1511,11 +1508,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1530,7 +1527,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1548,7 +1545,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1585,7 +1582,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1601,7 +1598,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..51748c99ad 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index f4247ea70d..f65d05c24c 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -631,6 +632,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1241,6 +1244,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..927ae319b1 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 106;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 619b2f9c71..9f1de1e42f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,8 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1a19921f80..4e137140bd 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -15,9 +15,11 @@
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,33 +43,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -78,9 +53,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -116,13 +90,6 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_blocks_hit;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -181,280 +148,32 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgBgWriter
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter buf_alloc;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -486,96 +205,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -584,13 +215,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9D
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -600,7 +227,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -610,29 +236,52 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ Oid databaseid;
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
+ LWLock lock; /* Lock for the above members */
+
+ /* non-shared members */
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
} PgStat_StatDBEntry;
+#define SHARED_DBENT_SIZE offsetof(PgStat_StatDBEntry, snapshot_tables)
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
Oid tableid;
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -651,19 +300,15 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -678,7 +323,7 @@ typedef struct PgStat_StatFuncEntry
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -694,7 +339,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -760,7 +405,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_ALL,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
@@ -1001,7 +645,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1198,13 +842,15 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1219,29 +865,26 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
-extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void pgstat_reset_all(void);
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
@@ -1402,8 +1045,8 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1412,11 +1055,14 @@ extern void pgstat_send_bgwriter(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_snapshot(PgStat_StatDBEntry *dbent, Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 8fda8e4f78..13371e8cb7 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -220,6 +220,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_SXACT,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 83a15f6795..77d1572a99 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.18.2
* [PATCH v27 5/7] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 500ms and such large data is
serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 500ms. Locks
on the shared statistics is acquired by the units of such like tables,
functions so the expected chance of collision are not so
high. Furthermore, until 1 second has elapsed since the last flushing
to shared stats, lock failure postpones stats flushing so that lock
contention doesn't slow down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/transam/xlog.c | 4 +-
src/backend/catalog/index.c | 24 +-
src/backend/postmaster/autovacuum.c | 54 +-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 12 +-
src/backend/postmaster/pgarch.c | 4 +-
src/backend/postmaster/pgstat.c | 4843 +++++++-----------
src/backend/postmaster/postmaster.c | 85 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 1 +
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/tcop/postgres.c | 26 +-
src/backend/utils/adt/pgstatfuncs.c | 53 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 +
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 2 +
src/include/pgstat.h | 514 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/timeout.h | 1 +
21 files changed, 2055 insertions(+), 3598 deletions(-)
inline patch 1: src/backend/access/transam/xlog.c, src/backend/catalog/index.c, src/backend/postmaster/autovacuum.c and 18 more (+2055/-3598 lines) download inline-1.patch
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 4da7ed3657..cee0572367 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8528,9 +8528,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 2d81bc3cbc..4de574ae00 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1687,28 +1687,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be sent by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Close relations */
table_close(pg_class, RowExclusiveLock);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index da75e755f0..c00b04a624 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -336,9 +336,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1936,8 +1933,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1956,12 +1951,6 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
@@ -2009,9 +1998,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2090,8 +2076,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2174,8 +2160,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2734,29 +2720,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2777,17 +2740,12 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
bool doanalyze;
autovac_table *tab = NULL;
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
bool wraparound;
AutoVacOpts *avopts;
/* use fresh stats */
autovac_refresh_stats();
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* fetch the relation's relcache entry */
classTup = SearchSysCacheCopy1(RELOID, ObjectIdGetDatum(relid));
if (!HeapTupleIsValid(classTup))
@@ -2811,8 +2769,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
}
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..94bdd664b5 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -236,7 +236,7 @@ BackgroundWriterMain(void)
/*
* Send off activity statistics to the stats collector
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..8a2fd0ddb2 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -350,7 +350,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ BgWriterStats.requested_checkpoints++;
}
/*
@@ -364,7 +364,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ BgWriterStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -492,7 +492,7 @@ CheckpointerMain(void)
* worth the trouble to split the stats support into two independent
* stats message types.)
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* Sleep until we are signaled or it's time for another checkpoint or
@@ -693,7 +693,7 @@ CheckpointWriteDelay(int flags, double progress)
/*
* Report interim activity statistics to the stats collector.
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1238,8 +1238,8 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ BgWriterStats.buf_written_backend += CheckpointerShmem->num_backend_writes;
+ BgWriterStats.buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 6fe7a136ba..f0b524ca50 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -401,7 +401,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we successfully
* archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
@@ -411,7 +411,7 @@ pgarch_ArchiverCopyLoop(void)
* Tell the collector about the WAL file that we failed to
* archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 4763c24be9..c0760854f4 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (1000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (250ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (10000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,68 +35,43 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
-#include "utils/timestamp.h"
/* ----------
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 1000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 250 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 10000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -116,7 +86,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -131,75 +100,162 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-
-/* ----------
- * Local data
- * ----------
+ * Shared stats bootstrap information, protected by StatsLock.
+ *
+ * refcount is used to know whether a process going to detach shared stats is
+ * the last process or not. The last process writes out the stats files.
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ dsa_pointer global_stats; /* DSA pointer to global stats */
+ dsa_pointer archiver_stats; /* Ditto for archiver stats */
+ int refcount; /* # of processes that is attaching the shared
+ * stats memory */
+} StatsShmemStruct;
-static time_t last_pgstat_start_time;
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
+ * Enums and types to define shared statistics structure.
+ *
+ * Statistics entries for each object is stored in individual DSA-allocated
+ * memory. Every entry is pointed from the dshash pgStatSharedHash via
+ * dsa_pointer. The structure makes object-stats entries not moved by dshash
+ * resizing, and allows the dshash can release lock sooner on stats
+ * updates. Also it reduces interfering among write-locks on each stat entry by
+ * not relying on partition lock of dshash. pgStatLocalHashEntry is the equivalent of pgStatSharedHash for local stat entries.
+ *
+ * Each stat entry is enveloped by the type PgStatEnvelope, which stores common
+ * attribute of all kind of statistics and a LWLock lock object.
*
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStatEnvelope (dsa memory block)
+ *
+ * Local stats are stored as:
+ *
+ * dshash pgStatLocalHash
+ * -> PgStatLocalHashEntry (dynahash entry)
+ * (direct pointer)-> PgStatEnvelope (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries. */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_ALL, /* Not a type, for the parameters of
+ * pgstat_collect_stat_entries */
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION, /* per-function statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static size_t pgstat_entsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry) /* PGSTAT_TYPE_FUNCTION */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static size_t pgstat_localentsize[] =
+{
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry) /* PGSTAT_TYPE_FUNCTION */
+};
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashEntryKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object OID */
+} PgStatHashEntryKey;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Stats numbers that are waiting for flushing out to shared stats are held in
+ * pgStatLocalHash,
*/
-static HTAB *pgStatFunctions = NULL;
+typedef struct PgStatHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ dsa_pointer env; /* pointer to shared stats envelope in DSA */
+} PgStatHashEntry;
+
+/* struct for shared statistics entry pointed from shared hash entry. */
+typedef struct PgStatEnvelope
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* databaseid */
+ Oid objectid; /* objectid */
+ size_t len; /* length of body, fixed per type. */
+ LWLock lock; /* lightweight lock to protect body */
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatEnvelope;
+
+#define PgStatEnvelopeSize(bodylen) \
+ (offsetof(PgStatEnvelope, body) + (bodylen))
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ PgStatEnvelope *env; /* pointer to stats envelope in heap */
+} PgStatLocalHashEntry;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * Snapshot is stats entry that is locally copied to offset stable values for a
+ * transaction.
*/
-static bool have_function_stats = false;
+typedef struct PgStatSnapshot
+{
+ PgStatHashEntryKey key;
+ bool negative;
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatSnapshot;
+
+#define PgStatSnapshotSize(bodylen) \
+ (offsetof(PgStatSnapshot, body) + (bodylen))
+
+/* parameter for shared hashes */
+static const dshash_parameters dsh_rootparams = {
+ sizeof(PgStatHashEntryKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
+
+/* Local stats numbers are stored here */
+static HTAB *pgStatLocalHash = NULL;
+
+#define HAVE_ANY_PENDING_STATS() \
+ (pgStatLocalHash != NULL || \
+ pgStatXactCommit != 0 || pgStatXactRollback != 0)
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -236,11 +292,10 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatSnapshotHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -249,19 +304,17 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are collected not per database nor per table
+ * basis. shared_* points to shared memory and snapshot_* are backend
+ * snapshots.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+static bool global_snapshot_is_valid = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats snapshot_globalStats;
/*
* Total time charged to functions so far in the current backend.
@@ -270,523 +323,269 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/*
+ * Newly created shared stats entries needs to be initialized before the other
+ * processes get access it. get_stat_entry() calls it for the purpose.
+ */
+typedef void (*entry_initializer) (PgStatEnvelope * env);
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStatEnvelope * get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait,
+ entry_initializer initfunc, bool *found);
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid);
+static void create_missing_dbentries(void);
+static void pgstat_write_database_stats(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry);
+
+static bool flush_tabstat(PgStatEnvelope * env, bool nowait);
+static bool flush_funcstat(PgStatEnvelope * env, bool nowait);
+static bool flush_dbstat(PgStatEnvelope * env, bool nowait);
+
+static void init_dbentry(PgStatEnvelope * env);
+static void init_funcentry(PgStatEnvelope * env);
+static void init_tabentry(PgStatEnvelope * env);
+
+static bool delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait);
+static PgStatEnvelope * get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static void pgstat_snapshot_global_stats(void);
+
static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
-
-static void pgstat_setup_memcxt(void);
-
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
+ bool found;
-#define TESTBYTEVAL ((char) 199)
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
+}
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgStatLocalContext and pgStatSnapshotContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgStatLocalContext)
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+
+ if (!pgStatSnapshotContext)
+ pgStatSnapshotContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Database statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+}
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read saved statistics files if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * Don't use dsm under postmaster, or when not tracking counts.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ pgstat_setup_memcxt();
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ if (area)
+ return;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
+ /*
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that processes going to exit can find whether it
+ * is the last or not.
+ */
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
+ {
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_rootparams, 0);
- test_byte++; /* just make sure variable is changed */
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* Load saved data if any. */
+ pgstat_read_statsfiles();
- /* If we get here, we have a working socket */
- break;
+ StatsShmem->refcount = 1;
}
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockRelease(StatsLock);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * If we're not the first process, attach existing shared stats area
+ * outside the StatsLock section.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (!area)
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* Attach shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_rootparams,
+ StatsShmem->hash_handle, 0);
+
+ /* Setup local variables */
+ pgStatLocalHash = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
}
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
- {
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
- }
-
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+ global_snapshot_is_valid = false;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_stats)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
- {
- int nchars;
- Oid tmp_oid;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount < 1)
+ {
/*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
+ * The process is the last one that is attaching the shared stats
+ * memory. Write out the stats files if requested.
*/
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
+ if (write_stats)
+ pgstat_write_statsfiles();
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ /* No one is using the area. */
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
- FreeDir(dir);
+
+ LWLockRelease(StatsLock);
+
+ /*
+ * Detach the area. Automatically destroyed when the last process detached
+ * it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatLocalHash = NULL;
+ global_snapshot_is_valid = false;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
-#ifdef EXEC_BACKEND
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
-
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
/*
- * Okay, fork off the collector.
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
+ detach_shared_stats(false); /* Don't write files. */
+ attach_shared_stats();
}
/* ------------------------------------------------------------
@@ -794,144 +593,479 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait = !force; /* Don't use force ever after */
+ HASH_SEQ_STATUS scan;
+ PgStatLocalHashEntry *lent;
+ PgStatLocalHashEntry **dbentlist;
+ int dbentlistlen = 8;
+ int ndbentries = 0;
+ int remains = 0;
int i;
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (area == NULL || !HAVE_ANY_PENDING_STATS())
+ return 0;
+
+ dbentlist = palloc(sizeof(PgStatLocalHashEntry *) * dbentlistlen);
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
+ if (nowait)
+ {
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
+ {
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ nowait = false;
+ }
/*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * flush_tabstat applies some of stats numbers of flushed entries into
+ * local database stats. So flush-out database stats later.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ if (pgStatLocalHash)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /* Step 1: flush out other than database stats */
+ hash_seq_init(&scan, pgStatLocalHash);
+ while ((lent = (PgStatLocalHashEntry *) hash_seq_search(&scan)) != NULL)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
+ bool remove = false;
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
+ switch (lent->env->type)
+ {
+ case PGSTAT_TYPE_DB:
+ if (ndbentries >= dbentlistlen)
+ {
+ dbentlistlen *= 2;
+ dbentlist = repalloc(dbentlist,
+ sizeof(PgStatLocalHashEntry *) *
+ dbentlistlen);
+ }
+ dbentlist[ndbentries++] = lent;
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent->env, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent->env, nowait))
+ remove = true;
+ break;
+ default:
+ Assert(false);
+ }
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
+ }
+
+ /* Step 2: flush out database stats */
+ for (i = 0; i < ndbentries; i++)
+ {
+ PgStatLocalHashEntry *lent = dbentlist[i];
+
+ if (flush_dbstat(lent->env, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ pfree(dbentlist);
+
+ if (remains <= 0)
+ {
+ hash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
+ }
+
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (shared_globalStats->stats_timestamp < now)
+ shared_globalStats->stats_timestamp = now;
+ LWLockRelease(StatsLock);
+
+ /*
+ * If we have pending local stats, let the caller know the retry interval.
+ */
+ if (HAVE_ANY_PENDING_STATS())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
+
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
+
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
+
+ return retry_interval / 1000;
}
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
+}
+
+/*
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_tabstat(PgStatEnvelope * lenv, bool nowait)
+{
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+ bool found;
+
+ Assert(lenv->type == PGSTAT_TYPE_TABLE);
+
+ lstats = (PgStat_TableStatus *) &lenv->body;
+ dboid = lstats->t_shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_TABLE, dboid, lstats->t_id,
+ nowait, init_tabentry, &found);
+
+ /* skip if dshash failed to acquire lock */
+ if (shenv == NULL)
+ return false;
+
+ /* retrieve the shared table stats entry from the envelope */
+ shtabstats = (PgStat_StatTabEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated ||
+ lstats->t_counts.vacuum_count > 0 ||
+ lstats->t_counts.analyze_count > 0 ||
+ lstats->t_counts.autovac_vacuum_count > 0 ||
+ lstats->t_counts.autovac_analyze_count > 0)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ /* clear the change counter if requested */
+ if (lstats->t_counts.reset_changed_tuples)
+ shtabstats->changes_since_analyze = 0;
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /*
+ * Update vacuum/analyze timestamp and counters, so that the values won't
+ * goes back.
+ */
+ if (shtabstats->vacuum_timestamp < lstats->vacuum_timestamp)
+ shtabstats->vacuum_timestamp = lstats->vacuum_timestamp;
+ shtabstats->vacuum_count += lstats->t_counts.vacuum_count;
+
+ if (shtabstats->autovac_vacuum_timestamp < lstats->autovac_vacuum_timestamp)
+ shtabstats->autovac_vacuum_timestamp = lstats->autovac_vacuum_timestamp;
+ shtabstats->autovac_vacuum_count += lstats->t_counts.autovac_vacuum_count;
+
+ if (shtabstats->analyze_timestamp < lstats->analyze_timestamp)
+ shtabstats->analyze_timestamp = lstats->analyze_timestamp;
+ shtabstats->analyze_count += lstats->t_counts.analyze_count;
+
+ if (shtabstats->autovac_analyze_timestamp < lstats->autovac_analyze_timestamp)
+ shtabstats->autovac_analyze_timestamp = lstats->autovac_analyze_timestamp;
+ shtabstats->autovac_analyze_count += lstats->t_counts.autovac_analyze_count;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shenv->lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ return true;
+}
+
+/* ----------
+ * init_tabentry() -
+ *
+ * initializes table stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_tabentry(PgStatEnvelope * env)
+{
+ PgStat_StatTabEntry *tabent = (PgStat_StatTabEntry *) &env->body;
+
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * If it's a new table entry, initialize counters to the values we just
+ * got.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
-
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ Assert(env->type == PGSTAT_TYPE_TABLE);
+ tabent->tableid = env->objectid;
+ tabent->numscans = 0;
+ tabent->tuples_returned = 0;
+ tabent->tuples_fetched = 0;
+ tabent->tuples_inserted = 0;
+ tabent->tuples_updated = 0;
+ tabent->tuples_deleted = 0;
+ tabent->tuples_hot_updated = 0;
+ tabent->n_live_tuples = 0;
+ tabent->n_dead_tuples = 0;
+ tabent->changes_since_analyze = 0;
+ tabent->blocks_fetched = 0;
+ tabent->blocks_hit = 0;
+
+ tabent->vacuum_timestamp = 0;
+ tabent->vacuum_count = 0;
+ tabent->autovac_vacuum_timestamp = 0;
+ tabent->autovac_vacuum_count = 0;
+ tabent->analyze_timestamp = 0;
+ tabent->analyze_count = 0;
+ tabent->autovac_analyze_timestamp = 0;
+ tabent->autovac_analyze_count = 0;
}
+
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * pgstat_flush_funcstat - flush out a local function stats entry
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatEnvelope * env, bool nowait)
+{
+ /* we assume this inits to all zeroes: */
+ static const PgStat_FunctionCounts all_zeroes;
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatFuncEntry *sharedent = NULL; /* shared stats entry */
+ bool found;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) &env->body;
+
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&localent->f_counts, &all_zeroes,
+ sizeof(PgStat_FunctionCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, localent->f_id,
+ nowait, init_funcentry, &found);
+ /* skip if dshash failed to acquire lock */
+ if (sharedent == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* retrieve the shared table stats entry from the envelope */
+ sharedent = (PgStat_StatFuncEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->f_numcalls += localent->f_counts.f_numcalls;
+ sharedent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ sharedent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shenv->lock);
+
+ return true;
+}
+
+
+/* ----------
+ * init_funcentry() -
+ *
+ * initializes function stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
*/
static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+init_funcentry(PgStatEnvelope * env)
{
- int n;
- int len;
+ PgStat_StatFuncEntry *shstat = (PgStat_StatFuncEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ shstat->functionid = env->objectid;
+ shstat->f_numcalls = 0;
+ shstat->f_total_time = 0;
+ shstat->f_self_time = 0;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatEnvelope * env, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStatEnvelope *shenv;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &env->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_DB, localent->databaseid, InvalidOid,
+ nowait, init_dbentry, NULL);
+
+ /* skip if dshash failed to acquire lock */
+ if (!shenv)
+ return false;
+
+ /* retrieve the shared stats entry from the envelope */
+ sharedent = (PgStat_StatDBEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(localent->databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -939,257 +1073,102 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&shenv->lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
+
+/* ----------
+ * init_dbentry() -
+ *
+ * initializes database stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_dbentry(PgStatEnvelope * env)
+{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+ dbentry->databaseid = env->databaseid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ /* initialize the new shared entry */
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
+}
+
+
/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+ * Create the filename for a DB stat file; filename is output parameter points
+ * to a character buffer of length len.
*/
static void
-pgstat_send_funcstats(void)
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
- {
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
-
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
}
/* ----------
- * pgstat_vacuum_stat() -
+ * collect_stat_entries() -
*
- * Will tell the collector about objects he can get rid of.
+ * Collect the shared statistics entries specified by type and dbid. Returns a
+ * list of pointer to shared statistics in palloc'ed memory. If type is
+ * PGSTAT_TYPE_ALL, all types of statistics of the database is collected. If
+ * type is PGSTAT_TYPE_DB, the parameter dbid is ignored and collect all
+ * PGSTAT_TYPE_DB entries.
* ----------
*/
-void
-pgstat_vacuum_stat(void)
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
- return;
-
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
-
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ int listlen = 16;
+ PgStatEnvelope **envlist = palloc(sizeof(PgStatEnvelope * *) * listlen);
+ int n = 0;
+
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ if ((type != PGSTAT_TYPE_ALL && p->key.type != type) ||
+ (type != PGSTAT_TYPE_DB && p->key.databaseid != dbid))
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
+ if (n >= listlen - 1)
{
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
+ listlen *= 2;
+ envlist = repalloc(envlist, listlen * sizeof(PgStatEnvelope * *));
}
+ envlist[n++] = dsa_get_address(area, p->env);
}
+ dshash_seq_term(&hstat);
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
+ envlist[n] = NULL;
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
- }
+ return envlist;
}
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
* into a temporary hash table. Caller should hash_destroy the result
@@ -1198,7 +1177,7 @@ pgstat_vacuum_stat(void)
* ----------
*/
static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
HTAB *htab;
HASHCTL hash_ctl;
@@ -1212,7 +1191,7 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
hash_ctl.entrysize = sizeof(Oid);
hash_ctl.hcxt = CurrentMemoryContext;
htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
+ PGSTAT_TABLE_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
@@ -1239,65 +1218,184 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
}
+/* ----------
+ * pgstat_vacuum_stat() -
+ *
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
+ * ----------
+ */
+void
+pgstat_vacuum_stat(void)
+{
+ HTAB *dbids; /* database ids */
+ HTAB *relids; /* relation ids in the current database */
+ HTAB *funcids; /* function ids in the current database */
+ PgStatEnvelope **victims; /* victim entry list */
+ int arraylen = 0; /* storage size of the above */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+ int i;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+
+ /* collect victims from shared stats */
+ arraylen = 16;
+ victims = palloc(sizeof(PgStatEnvelope * *) * arraylen);
+ nvictims = 0;
+
+ dshash_seq_init(&dshstat, pgStatSharedHash, false);
+
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ HTAB *oidtab;
+ Oid *key;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
+ continue;
+
+ switch (ent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidtab = dbids;
+ key = &ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidtab = relids;
+ key = &ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidtab = funcids;
+ key = &ent->key.objectid;
+ break;
+ case PGSTAT_TYPE_ALL:
+ Assert(false);
+ break;
+ }
+
+ /* Skip existent objects. */
+ if (hash_search(oidtab, key, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* extend the list if needed */
+ if (nvictims >= arraylen)
+ {
+ arraylen *= 2;
+ victims = repalloc(victims, sizeof(PgStatEnvelope * *) * arraylen);
+ }
+
+ victims[nvictims++] = dsa_get_address(area, ent->env);
+ }
+ dshash_seq_term(&dshstat);
+ hash_destroy(dbids);
+ hash_destroy(relids);
+ hash_destroy(funcids);
+
+ /* Now try removing the victim entries */
+ for (i = 0; i < nvictims; i++)
+ {
+ PgStatEnvelope *p = victims[i];
+
+ delete_stat_entry(p->type, p->databaseid, p->objectid, true);
+ }
+}
+
+
+/* ----------
+ * delete_stat_entry -
+ *
+ * Deletes the specified entry from shared stats hash
+ *
+ * Returns true when successfully deleted.
+ * ----------
+ */
+static bool
+delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait)
+{
+ PgStatHashEntryKey key;
+ PgStatHashEntry *ent;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ ent = dshash_find_extended(pgStatSharedHash, &key,
+ true, nowait, false, NULL);
+
+ if (!ent)
+ return false; /* lock failed or not found */
+
+ /* The entry is exclusively locked, so we can free the chunk first. */
+ dsa_free(area, ent->env);
+ dshash_delete_entry(pgStatSharedHash, ent);
+
+ return true;
+}
+
+
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
+ Assert(OidIsValid(databaseid));
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
+ for (p = envlist; *p != NULL; p++)
+ delete_stat_entry((*p)->type, (*p)->databaseid, (*p)->objectid, true);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ pfree(envlist);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1306,20 +1404,47 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ /* Lookup the entries of the current database in the stats hash. */
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
+ for (p = envlist; *p != NULL; p++)
+ {
+ PgStatEnvelope *env = *p;
+ PgStat_StatDBEntry *dbstat;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+
+ switch (env->type)
+ {
+ case PGSTAT_TYPE_TABLE:
+ init_tabentry(env);
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ init_funcentry(env);
+ break;
+
+ case PGSTAT_TYPE_DB:
+ init_dbentry(env);
+ dbstat = (PgStat_StatDBEntry *) &env->body;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ break;
+ default:
+ Assert(false);
+ }
+
+ LWLockRelease(&env->lock);
+ }
+
+ pfree(envlist);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1328,29 +1453,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ /* Reset the bgwriter statistics for the cluster. */
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1359,17 +1492,42 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ env = get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid,
+ false, NULL, NULL);
+ Assert(env);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&env->lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ env = get_stat_entry(stattype, MyDatabaseId, objoid, false, NULL, NULL);
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ if (env->type == PGSTAT_TYPE_TABLE)
+ init_tabentry(env);
+ else
+ {
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ init_funcentry(env);
+ }
+ LWLockRelease(&env->lock);
}
/* ----------
@@ -1383,48 +1541,63 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ ts = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Store the last autovacuum time in the database's hash table entry.
+ */
+ dbentry = get_local_dbstat_entry(dboid);
+ dbentry->last_autovac_time = ts;
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_TableStatus *tabentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+ tabentry = get_local_tabstat_entry(tableoid, shared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->t_counts.autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->t_counts.vacuum_count++;
+ }
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1435,9 +1608,10 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_TableStatus *tabentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1445,10 +1619,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1466,158 +1640,172 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ tabentry = get_local_tabstat_entry(RelationGetRelid(rel),
+ rel->rd_rel->relisshared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->t_counts.reset_changed_tuples = true;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.analyze_count++;
+ }
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-
-/* --------
- * pgstat_report_checksum_failures_in_db() -
- *
- * Tell the collector about one or more checksum failures.
- * --------
- */
-void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
-{
- PgStat_MsgChecksumFailure msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
-
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
-/* ----------
- * pgstat_ping() -
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Send some junk data to the collector to increase traffic.
- * ----------
+ * Reports about one or more checksum failures.
+ * --------
*/
void
-pgstat_ping(void)
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgDummy msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dbentry = get_local_dbstat_entry(dboid);
+
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
{
+ PgStatEnvelope *env;
PgStat_BackendFunctionEntry *htabent;
bool found;
@@ -1628,26 +1816,15 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
+ htabent = (PgStat_BackendFunctionEntry *) &env->body;
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
if (!found)
MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent->f_id = fcinfo->flinfo->fn_oid;
+
fcu->fs = &htabent->f_counts;
/* save stats for this function, later used to compensate for recursion */
@@ -1660,31 +1837,38 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
+ PgStatEnvelope *env;
+
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+ if (!env)
return NULL;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ return (PgStat_BackendFunctionEntry *) &env->body;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1725,9 +1909,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1739,8 +1920,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1760,7 +1940,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1776,116 +1957,157 @@ pgstat_initstats(Relation rel)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
-/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize body part of the returned envelope.
+ * ----------
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+static PgStatEnvelope *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
+ PgStatHashEntryKey key;
+ PgStatLocalHashEntry *entry;
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ if (pgStatLocalHash == NULL)
{
HASHCTL ctl;
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = sizeof(PgStatLocalHashEntry);
+
+ pgStatLocalHash = hash_create("Local stat entries",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ entry = hash_search(pgStatLocalHash, &key,
+ create ? HASH_ENTER : HASH_FIND, found);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ {
+ int len = pgstat_localentsize[type];
+
+ entry->env = MemoryContextAlloc(CacheMemoryContext,
+ PgStatEnvelopeSize(len));
+ entry->env->type = type;
+ entry->env->len = len;
}
+ return entry->env;
+}
+
+/* ----------
+ * get_local_dbstat_entry() -
+ *
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
+ */
+static PgStat_StatDBEntry *
+get_local_dbstat_entry(Oid dbid)
+{
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ bool found;
+
/*
* Find an entry or create a new one.
*/
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
+ env = get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
if (!found)
{
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
+ dbentry->databaseid = dbid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
}
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
+ return dbentry;
+}
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
+ */
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
+{
+ PgStatEnvelope *env;
+ PgStat_TableStatus *tabentry;
+ bool found;
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- return entry;
+ tabentry = (PgStat_TableStatus *) &env->body;
+
+ if (!found)
+ {
+ tabentry->t_id = rel_id;
+ tabentry->t_shared = isshared;
+ tabentry->trans = NULL;
+ MemSet(&tabentry->t_counts, 0, sizeof(PgStat_TableCounts));
+ tabentry->vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ }
+
+ return tabentry;
}
-/*
- * find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
+
+/* ----------
+ * find_tabstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel from the current
+ * database then from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry, return NULL, don't create a new one.
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStatEnvelope *env;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!env)
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
+ if (env)
+ return (PgStat_TableStatus *) &env->body;
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return NULL;
}
/*
@@ -2362,7 +2584,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2398,7 +2620,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2415,88 +2637,176 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
}
+/* ----------
+ * snapshot_statentry() -
+ *
+ * Common routine for functions pgstat_fetch_stat_*entry()
+ *
+ * Returns the pointer to the snapshot of the shared entry for the key or NULL
+ * if not found. Returned snapshots are stable during the current transaction
+ * or until pgstat_clear_snapshot() is called.
+ *
+ * Created snapshots are stored in pgStatSnapshotHash.
+ */
+static void *
+snapshot_statentry(const PgStatTypes type, const Oid dbid, const Oid objid)
+{
+ PgStatSnapshot *snap = NULL;
+ bool found;
+ PgStatHashEntryKey key;
+ size_t statentsize = pgstat_entsize[type];
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ /*
+ * Create new hash, with rather arbitrary initial number of entries since
+ * we don't know how this hash will grow.
+ */
+ if (!pgStatSnapshotHash)
+ {
+ HASHCTL ctl;
+
+ /*
+ * Create the hash in the stats context
+ *
+ * The entry is prepended by common header part represented by
+ * PgStatSnapshot.
+ */
+
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = PgStatSnapshotSize(statentsize);
+ ctl.hcxt = pgStatSnapshotContext;
+ pgStatSnapshotHash = hash_create("pgstat snapshot hash", 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ /* Find a snapshot */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ snap = hash_search(pgStatSnapshotHash, &key, HASH_ENTER, &found);
+
+ /*
+ * Refer shared hash if not found in the snapshot hash.
+ *
+ * In transaction state, it is obvious that we should create a snapshot
+ * entriy for consistency. If we are not, we return an up-to-date entry.
+ * Having said that, we need a snapshot since shared stats entry can be
+ * modified anytime. We share the same snapshot entry for the purpose.
+ */
+ if (!found || !IsTransactionState())
+ {
+ PgStatEnvelope *shenv;
+
+ shenv = get_stat_entry(type, dbid, objid, true, NULL, NULL);
+
+ if (shenv)
+ memcpy(&snap->body, &shenv->body, statentsize);
+
+ snap->negative = !shenv;
+ }
+
+ if (snap->negative)
+ return NULL;
+
+ return &snap->body;
+}
+
+
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *)
+ snapshot_statentry(PGSTAT_TYPE_DB, dbid, InvalidOid);
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_snapshot() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_snapshot(bool shared, Oid reloid)
+{
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(PGSTAT_TYPE_TABLE, dboid, reloid);
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2510,24 +2820,48 @@ pgstat_fetch_stat_tabentry(Oid relid)
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
-
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
-
- return funcentry;
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, func_id);
}
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ * ----------
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+
+ attach_shared_stats();
+
+ /* Nothing to do if already done */
+ if (global_snapshot_is_valid)
+ return;
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+
+ memcpy(&snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ LWLockRelease(StatsLock);
+
+ global_snapshot_is_valid = true;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
+}
/* ----------
* pgstat_fetch_stat_beentry() -
@@ -2599,9 +2933,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return &snapshot_archiverStats;
}
@@ -2616,9 +2951,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return &snapshot_globalStats;
}
@@ -2832,8 +3168,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3009,12 +3345,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3027,7 +3366,7 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
@@ -3044,6 +3383,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3304,7 +3645,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3599,9 +3941,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
break;
@@ -4230,94 +4569,71 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Report archiver statistics
* ----------
*/
void
-pgstat_send_archiver(const char *xlog, bool failed)
+pgstat_report_archiver(const char *xlog, bool failed)
{
- PgStat_MsgArchiver msg;
+ TimestampTz now = GetCurrentTimestamp();
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- StrNCpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ if (failed)
+ {
+ /* Failed archival attempt */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ /* Successful archival operation */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *l = &BgWriterStats;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += l->timed_checkpoints;
+ shared_globalStats->requested_checkpoints += l->requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += l->checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += l->checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += l->buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += l->buf_written_clean;
+ shared_globalStats->maxwritten_clean += l->maxwritten_clean;
+ shared_globalStats->buf_written_backend += l->buf_written_backend;
+ shared_globalStats->buf_fsync_backend += l->buf_fsync_backend;
+ shared_globalStats->buf_alloc += l->buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4326,424 +4642,30 @@ pgstat_send_bgwriter(void)
}
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE,
- pgStatSock, -1L,
- WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitLatchOrSocket(MyLatch,
- WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_SOCKET_READABLE | WL_TIMEOUT,
- pgStatSock,
- 2 * 1000L /* msec */ ,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr & WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
-{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
-
- return result;
-}
-
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
-{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
-}
-
-
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
elog(DEBUG2, "writing stats file \"%s\"", statfile);
+ create_missing_dbentries();
+
/*
* Open the statistics temp file to write out the current values.
*/
@@ -4760,7 +4682,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4772,32 +4694,31 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_DB, InvalidOid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &(*penv)->body;
+
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_database_stats(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
@@ -4808,6 +4729,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
+
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4840,55 +4763,19 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
}
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
+ * pgstat_write_database_stats() -
+ * Write the stat file for a single database.
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_database_stats(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -4896,8 +4783,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -4924,24 +4811,31 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_TABLE, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatTabEntry *tabentry = (PgStat_StatTabEntry *) &(*penv)->body;
+
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_FUNCTION, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatFuncEntry *funcentry =
+ (PgStat_StatFuncEntry *) &(*penv)->body;
+
fputc('F', fpout);
rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* No more output to be done. Close the temp file and replace the old
@@ -4975,94 +4869,165 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
+}
- if (permanent)
+/* ----------
+ * create_missing_dbentries() -
+ *
+ * There may be the case where database entry is missing for the database
+ * where object stats are recorded. This function creates such missing
+ * dbentries so that so that all stats entries can be written out to files.
+ * ----------
+ */
+static void
+create_missing_dbentries(void)
+{
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ HTAB *oidhash;
+ HASHCTL ctl;
+ HASH_SEQ_STATUS scan;
+ Oid *poid;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = CurrentMemoryContext;
+ oidhash = hash_create("Temporary table of OIDs",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Collect OID from the shared stats hash */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ hash_search(oidhash, &p->key.databaseid, HASH_ENTER, NULL);
+ dshash_seq_term(&hstat);
+
+ /* Create missing database entries if not exists. */
+ hash_seq_init(&scan, oidhash);
+ while ((poid = (Oid *) hash_seq_search(&scan)) != NULL)
+ (void) get_stat_entry(PGSTAT_TYPE_DB, *poid, InvalidOid,
+ false, init_dbentry, NULL);
+
+ hash_destroy(oidhash);
+}
+
+
+/* ----------
+ * get_stat_entry() -
+ *
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new envelope. If found is not NULL, it is set to true if
+ * existing entry is found or false if not.
+ * ----------
+ */
+static PgStatEnvelope *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait, entry_initializer initfunc, bool *found)
+{
+ bool create = (initfunc != NULL);
+ PgStatHashEntry *shent;
+ PgStatEnvelope *shenv = NULL;
+ PgStatHashEntryKey key;
+ bool myfound;
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ shent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &myfound);
+ if (shent)
{
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
+ if (create && !myfound)
+ {
+ /* Create new stats envelope. */
+ size_t envsize = PgStatEnvelopeSize(pgstat_entsize[type]);
+ dsa_pointer chunk = dsa_allocate0(area, envsize);
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
+ shenv = dsa_get_address(area, chunk);
+ shenv->type = type;
+ shenv->databaseid = dbid;
+ shenv->objectid = objid;
+ shenv->len = pgstat_entsize[type];
+ LWLockInitialize(&shenv->lock, LWTRANCHE_STATS);
+
+ /*
+ * The lock on dshsh is released just after. Call initializer
+ * callback before it is exposed to other process.
+ */
+ if (initfunc)
+ initfunc(shenv);
+
+ /* Link the new entry from the hash entry. */
+ shent->env = chunk;
+ }
+ else
+ shenv = dsa_get_address(area, shent->env);
+
+ dshash_release_lock(pgStatSharedHash, shent);
}
+
+ if (found)
+ *found = myfound;
+
+ return shenv;
}
+
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * Reads in existing activity statistics files into the shared stats hash.
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
+ PgStatEnvelope *env;
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5071,7 +5036,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5079,38 +5044,30 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5124,7 +5081,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
fpin) != offsetof(PgStat_StatDBEntry, tables))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5133,76 +5090,33 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+
+ env = get_stat_entry(PGSTAT_TYPE_DB, dbbuf.databaseid,
+ InvalidOid,
+ false, init_dbentry, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ /* don't allow duplicate dbentries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, tables));
+ /* Read the data from the database-specific file. */
+ pgstat_read_db_statsfile(dbentry);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5212,59 +5126,50 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
/* ----------
* pgstat_read_db_statsfile() -
*
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry)
{
+ PgStatEnvelope *env;
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
FILE *fpin;
int32 format_id;
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5277,14 +5182,14 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5297,25 +5202,21 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE,
+ dbentry->databaseid, tabbuf.tableid,
+ false, init_tabentry, &found);
+ tabentry = (PgStat_StatTabEntry *) &env->body;
+ /* don't allow duplicate entries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5331,25 +5232,20 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE, dbentry->databaseid,
+ funcbuf.functionid,
+ false, init_funcentry, &found);
+ funcentry = (PgStat_StatFuncEntry *) &env->body;
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5365,7 +5261,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5373,292 +5269,15 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
-done:
FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
-}
-
-/* ----------
- * pgstat_setup_memcxt() -
- *
- * Create pgStatLocalContext, if not already done.
- * ----------
- */
-static void
-pgstat_setup_memcxt(void)
-{
- if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
@@ -5677,741 +5296,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
}
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
+ if (pgStatSnapshotContext)
{
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
+ MemoryContextReset(pgStatSnapshotContext);
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
+ /* Reset variables that pointed to the context */
+ global_snapshot_is_valid = false;
+ pgStatSnapshotHash = NULL;
}
}
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signalling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
-}
-
/*
* Convert a potentially unsafely truncated activity string (see
* PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index fab4a9dd51..d418fe3bd0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -255,7 +255,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -503,7 +502,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1326,12 +1324,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1780,11 +1772,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2694,8 +2681,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3058,8 +3043,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3126,13 +3109,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3205,22 +3181,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3681,22 +3641,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3892,8 +3836,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3928,8 +3870,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4130,8 +4071,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5109,18 +5048,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5239,12 +5166,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6139,7 +6060,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6195,8 +6115,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6431,7 +6349,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index e05e2b3456..26414dadb2 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1947,7 +1947,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ BgWriterStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2057,7 +2057,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2247,7 +2247,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2255,7 +2255,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 427b0d59cd..58a442f482 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 4c14e51c67..f61fd3e8ad 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -523,6 +523,7 @@ RegisterLWLockTranches(void)
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
+ LWLockRegisterTranche(LWTRANCHE_STATS, "activity_statistics");
/* Register named tranches. */
for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index db47843229..97eccb35d3 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -49,3 +49,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
LogicalRepWorkerLock 43
CLogTruncationLock 44
+StatsLock 45
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 00c77b66c7..e2998f965e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3189,6 +3189,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3763,6 +3769,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4201,6 +4208,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4213,8 +4222,13 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4249,7 +4263,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4257,6 +4271,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index cea01534a5..a1304dc3ce 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -33,9 +33,6 @@
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1244,7 +1241,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1260,7 +1257,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1276,7 +1273,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1292,7 +1289,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1308,7 +1305,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1324,7 +1321,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1340,7 +1337,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1356,7 +1353,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1372,7 +1369,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1405,7 +1402,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1421,7 +1418,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1436,7 +1433,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1451,7 +1448,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1466,7 +1463,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1481,7 +1478,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1496,7 +1493,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1511,11 +1508,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1530,7 +1527,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1548,7 +1545,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1585,7 +1582,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1601,7 +1598,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..51748c99ad 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index f4247ea70d..f65d05c24c 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -631,6 +632,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1241,6 +1244,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3c70499feb..927ae319b1 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 107;
+use Test::More tests => 106;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -123,7 +123,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 619b2f9c71..9f1de1e42f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,8 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index a07012bf4b..6fad13c4be 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -15,9 +15,11 @@
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,33 +43,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -78,9 +53,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -111,18 +85,17 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_delta_live_tuples;
PgStat_Counter t_delta_dead_tuples;
PgStat_Counter t_changed_tuples;
+ bool reset_changed_tuples;
PgStat_Counter t_blocks_fetched;
PgStat_Counter t_blocks_hit;
+
+ PgStat_Counter vacuum_count;
+ PgStat_Counter autovac_vacuum_count;
+ PgStat_Counter analyze_count;
+ PgStat_Counter autovac_analyze_count;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -156,6 +129,10 @@ typedef struct PgStat_TableStatus
Oid t_id; /* table's OID */
bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
} PgStat_TableStatus;
@@ -181,280 +158,32 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgTempFile
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter buf_alloc;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -486,96 +215,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -584,13 +225,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9D
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -600,7 +237,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -610,29 +246,55 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ Oid databaseid;
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/* ----------
+ * PgStat_HashMountInfo dshash mount information
+ * ----------
+ */
+typedef struct PgStat_HashMountInfo
+{
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
+} PgStat_HashMountInfo;
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
Oid tableid;
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -651,19 +313,15 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -678,7 +336,7 @@ typedef struct PgStat_StatFuncEntry
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -694,7 +352,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -760,7 +418,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1004,7 +661,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1201,13 +858,15 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1222,29 +881,26 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_report_autovac(Oid dboid);
@@ -1405,8 +1061,8 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1415,11 +1071,15 @@ extern void pgstat_send_bgwriter(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_snapshot(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 8fda8e4f78..13371e8cb7 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -220,6 +220,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_SXACT,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 83a15f6795..77d1572a99 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.18.2
* Re: shared-memory based stats collector
@ 2020-09-03 17:16 Stephen Frost <sfrost@snowman.net>
2020-09-07 02:04 ` Re: shared-memory based stats collector Michael Paquier <michael@paquier.xyz>
2020-09-07 13:46 ` Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
2020-09-08 08:01 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 3 replies; 17+ messages in thread
From: Stephen Frost @ 2020-09-03 17:16 UTC (permalink / raw)
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>; +Cc: alvherre@2ndquadrant.com; andres@anarazel.de; michael@paquier.xyz; thomas.munro@gmail.com; tomas.vondra@2ndquadrant.com; ah@cybertec.at; magnus@hagander.net; robertmhaas@gmail.com; tgl@sss.pgh.pa.us; pgsql-hackers
Greetings,
* Kyotaro Horiguchi (horikyota.ntt@gmail.com) wrote:
> At Mon, 01 Jun 2020 18:00:01 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > Rebased on the current HEAD. 36ac359d36 conflicts with this. Tranche
>
> Hmm. This conflicts with 0fd2a79a63. Reabsed on it.
Thanks for working on this and keeping it updated!
I've started taking a look and at least right off...
> >From 4926e50e7635548f86dcd0d36cbf56d168a5d242 Mon Sep 17 00:00:00 2001
> From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
> Date: Mon, 16 Mar 2020 17:15:35 +0900
> Subject: [PATCH v35 1/7] Use standard crash handler in archiver.
>
> The commit 8e19a82640 changed SIGQUIT handler of almost all processes
> not to run atexit callbacks for safety. Archiver process should behave
> the same way for the same reason. Exit status changes 1 to 2 but that
> doesn't make any behavioral change.
Shouldn't this:
a) be back-patched, as the other change was
b) also include a change to have the stats collector (which I realize is
removed later on in this patch set, but we're talking about fixing
existing things..) for the same reason, and because there isn't much
point in trying to write out the stats after we get a SIGQUIT, since
we're just going to blow them away again since we're going to go
through crash recovery..?
Might be good to have a separate thread to address these changes.
I've looked through (some of) this thread and through the patches also
and hope to provide a review of the bits that should be targetting v14
(unlike the above) soon.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: shared-memory based stats collector
2020-09-03 17:16 Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
@ 2020-09-07 02:04 ` Michael Paquier <michael@paquier.xyz>
2 siblings, 0 replies; 17+ messages in thread
From: Michael Paquier @ 2020-09-07 02:04 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>; alvherre@2ndquadrant.com; andres@anarazel.de; thomas.munro@gmail.com; tomas.vondra@2ndquadrant.com; ah@cybertec.at; magnus@hagander.net; robertmhaas@gmail.com; tgl@sss.pgh.pa.us; pgsql-hackers
On Thu, Sep 03, 2020 at 01:16:59PM -0400, Stephen Frost wrote:
> Shouldn't this:
>
> a) be back-patched, as the other change was
0001 is just a piece of refactoring, so I see no strong argument in
favor of a backpatch, IMHO.
> b) also include a change to have the stats collector (which I realize is
> removed later on in this patch set, but we're talking about fixing
> existing things..) for the same reason, and because there isn't much
> point in trying to write out the stats after we get a SIGQUIT, since
> we're just going to blow them away again since we're going to go
> through crash recovery..?
>
> Might be good to have a separate thread to address these changes.
>
> I've looked through (some of) this thread and through the patches also
> and hope to provide a review of the bits that should be targetting v14
> (unlike the above) soon.
The latest patch set fails to apply and the CF bot is complaining, so
a rebase is needed. I have switched the patch as waiting on author.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: shared-memory based stats collector
2020-09-03 17:16 Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
@ 2020-09-07 13:46 ` Stephen Frost <sfrost@snowman.net>
2 siblings, 0 replies; 17+ messages in thread
From: Stephen Frost @ 2020-09-07 13:46 UTC (permalink / raw)
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>; Michael Paquier <michael@paquier.xyz>; +Cc: alvherre@2ndquadrant.com; andres@anarazel.de; thomas.munro@gmail.com; tomas.vondra@2ndquadrant.com; ah@cybertec.at; magnus@hagander.net; robertmhaas@gmail.com; tgl@sss.pgh.pa.us; pgsql-hackers
Greetings,
* Stephen Frost (sfrost@snowman.net) wrote:
> * Kyotaro Horiguchi (horikyota.ntt@gmail.com) wrote:
> > The commit 8e19a82640 changed SIGQUIT handler of almost all processes
> > not to run atexit callbacks for safety. Archiver process should behave
> > the same way for the same reason. Exit status changes 1 to 2 but that
> > doesn't make any behavioral change.
>
> Shouldn't this:
>
> a) be back-patched, as the other change was
> b) also include a change to have the stats collector (which I realize is
> removed later on in this patch set, but we're talking about fixing
> existing things..) for the same reason, and because there isn't much
> point in trying to write out the stats after we get a SIGQUIT, since
> we're just going to blow them away again since we're going to go
> through crash recovery..?
* Michael Paquier (michael@paquier.xyz) wrote:
> 0001 is just a piece of refactoring, so I see no strong argument in
> favor of a backpatch, IMHO.
No, 0001 changes the SIGQUIT handler for the archiver process, which is
what 8e19a82640 was about changing for a bunch of the other subprocesses
and which was back-patched all the way, so I'm a bit confused about why
you're saying it's just refactoring..?
Note that exit() and _exit() aren't the same, and the latter is what's
now being used in SignalHandlerForCrashExit.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: shared-memory based stats collector
2020-09-03 17:16 Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
@ 2020-09-08 08:01 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2020-09-08 08:55 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2 siblings, 1 reply; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-09-08 08:01 UTC (permalink / raw)
To: sfrost@snowman.net; +Cc: alvherre@2ndquadrant.com; andres@anarazel.de; michael@paquier.xyz; thomas.munro@gmail.com; tomas.vondra@2ndquadrant.com; ah@cybertec.at; magnus@hagander.net; robertmhaas@gmail.com; tgl@sss.pgh.pa.us; pgsql-hackers
At Thu, 3 Sep 2020 13:16:59 -0400, Stephen Frost <sfrost@snowman.net> wrote in
> Greetings,
>
> * Kyotaro Horiguchi (horikyota.ntt@gmail.com) wrote:
> > At Mon, 01 Jun 2020 18:00:01 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > > Rebased on the current HEAD. 36ac359d36 conflicts with this. Tranche
> >
> > Hmm. This conflicts with 0fd2a79a63. Reabsed on it.
>
> Thanks for working on this and keeping it updated!
>
> I've started taking a look and at least right off...
>
> > >From 4926e50e7635548f86dcd0d36cbf56d168a5d242 Mon Sep 17 00:00:00 2001
> > From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
> > Date: Mon, 16 Mar 2020 17:15:35 +0900
> > Subject: [PATCH v35 1/7] Use standard crash handler in archiver.
> >
> > The commit 8e19a82640 changed SIGQUIT handler of almost all processes
> > not to run atexit callbacks for safety. Archiver process should behave
> > the same way for the same reason. Exit status changes 1 to 2 but that
> > doesn't make any behavioral change.
>
> Shouldn't this:
>
> a) be back-patched, as the other change was
> b) also include a change to have the stats collector (which I realize is
> removed later on in this patch set, but we're talking about fixing
> existing things..) for the same reason, and because there isn't much
> point in trying to write out the stats after we get a SIGQUIT, since
> we're just going to blow them away again since we're going to go
> through crash recovery..?
>
> Might be good to have a separate thread to address these changes.
+1
> I've looked through (some of) this thread and through the patches also
> and hope to provide a review of the bits that should be targetting v14
> (unlike the above) soon.
Thanks. Current the patch is found to lead to write contention heavier
than the current stats collector when nearly a thousand backend
heavily write to the same table. I need to address that.
- Collect stats via sockets (in the same way as the current implement)
and write/read to/from shared memory.
- Use our own lock-free (maybe) ring-buffer before stats-write enters
lock-waiting mode, then new stats collector(!) process consumes the
queue.
- Some other measures..
Anyway, I'll post a rebased version soon.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: shared-memory based stats collector
2020-09-03 17:16 Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
2020-09-08 08:01 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
@ 2020-09-08 08:55 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2020-09-09 08:01 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-09-08 08:55 UTC (permalink / raw)
To: sfrost@snowman.net; +Cc: alvherre@2ndquadrant.com; andres@anarazel.de; michael@paquier.xyz; thomas.munro@gmail.com; tomas.vondra@2ndquadrant.com; ah@cybertec.at; magnus@hagander.net; robertmhaas@gmail.com; tgl@sss.pgh.pa.us; pgsql-hackers
At Tue, 08 Sep 2020 17:01:47 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> At Thu, 3 Sep 2020 13:16:59 -0400, Stephen Frost <sfrost@snowman.net> wrote in> > I've looked through (some of) this thread and through the patches also
> > and hope to provide a review of the bits that should be targetting v14
> > (unlike the above) soon.
>
> Thanks. Current the patch is found to lead to write contention heavier
> than the current stats collector when nearly a thousand backend
> heavily write to the same table. I need to address that.
>
> - Collect stats via sockets (in the same way as the current implement)
> and write/read to/from shared memory.
>
> - Use our own lock-free (maybe) ring-buffer before stats-write enters
> lock-waiting mode, then new stats collector(!) process consumes the
> queue.
>
> - Some other measures..
>
> Anyway, I'll post a rebased version soon.
This is that. I'll continue seeking a way to solve the contention.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachments: download all (7 files, .tar.gz)
v36-0001-Use-standard-crash-handler-in-archiver.patch (text/x-patch, 1.7K) download
From 0eb08bfeea9a20d50c5391edee600f420acfa4b9 Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Mon, 16 Mar 2020 17:15:35 +0900
Subject: [PATCH v36 1/7] Use standard crash handler in archiver.
The commit 8e19a82640 changed SIGQUIT handler of almost all processes
not to run atexit callbacks for safety. Archiver process should behave
the same way for the same reason. Exit status changes 1 to 2 but that
doesn't make any behavioral change.
---
src/backend/postmaster/pgarch.c | 11 +----------
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 01ffd6513c..37be0e2bbb 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -96,7 +96,6 @@ static pid_t pgarch_forkexec(void);
#endif
NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
static void pgarch_MainLoop(void);
@@ -229,7 +228,7 @@ PgArchiverMain(int argc, char *argv[])
pqsignal(SIGHUP, SignalHandlerForConfigReload);
pqsignal(SIGINT, SIG_IGN);
pqsignal(SIGTERM, SignalHandlerForShutdownRequest);
- pqsignal(SIGQUIT, pgarch_exit);
+ pqsignal(SIGQUIT, SignalHandlerForCrashExit);
pqsignal(SIGALRM, SIG_IGN);
pqsignal(SIGPIPE, SIG_IGN);
pqsignal(SIGUSR1, pgarch_waken);
@@ -246,14 +245,6 @@ PgArchiverMain(int argc, char *argv[])
exit(0);
}
-/* SIGQUIT signal handler for archiver process */
-static void
-pgarch_exit(SIGNAL_ARGS)
-{
- /* SIGQUIT means curl up and die ... */
- exit(1);
-}
-
/* SIGUSR1 signal handler for archiver process */
static void
pgarch_waken(SIGNAL_ARGS)
--
2.18.4
v36-0002-sequential-scan-for-dshash.patch (text/x-patch, 8.4K) download
From 4a83b0e858f695ab855c3c1e184b8f235ee65521 Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Fri, 13 Mar 2020 16:58:03 +0900
Subject: [PATCH v36 2/7] sequential scan for dshash
Dshash did not allow scan the all entries sequentially. This adds the
functionality. The interface is similar but a bit different both from
that of dynahash and simple dshash search functions. One of the most
significant differences is the sequential scan interface of dshash
always needs a call to dshash_seq_term when scan ends. Another is
locking. Dshash holds partition lock when returning an entry,
dshash_seq_next() also holds lock when returning an entry but callers
shouldn't release it, since the lock is essential to continue a
scan. The seqscan interface allows entry deletion while a scan. The
in-scan deletion should be performed by dshash_delete_current().
---
src/backend/lib/dshash.c | 149 ++++++++++++++++++++++++++++++++++++++-
src/include/lib/dshash.h | 21 ++++++
2 files changed, 169 insertions(+), 1 deletion(-)
diff --git a/src/backend/lib/dshash.c b/src/backend/lib/dshash.c
index 78ccf03217..1ef093e2e9 100644
--- a/src/backend/lib/dshash.c
+++ b/src/backend/lib/dshash.c
@@ -127,6 +127,10 @@ struct dshash_table
#define NUM_SPLITS(size_log2) \
(size_log2 - DSHASH_NUM_PARTITIONS_LOG2)
+/* How many buckets are there in a given size? */
+#define NUM_BUCKETS(size_log2) \
+ (((size_t) 1) << (size_log2))
+
/* How many buckets are there in each partition at a given size? */
#define BUCKETS_PER_PARTITION(size_log2) \
(((size_t) 1) << NUM_SPLITS(size_log2))
@@ -153,6 +157,10 @@ struct dshash_table
#define BUCKET_INDEX_FOR_PARTITION(partition, size_log2) \
((partition) << NUM_SPLITS(size_log2))
+/* Choose partition based on bucket index. */
+#define PARTITION_FOR_BUCKET_INDEX(bucket_idx, size_log2) \
+ ((bucket_idx) >> NUM_SPLITS(size_log2))
+
/* The head of the active bucket for a given hash value (lvalue). */
#define BUCKET_FOR_HASH(hash_table, hash) \
(hash_table->buckets[ \
@@ -324,7 +332,7 @@ dshash_destroy(dshash_table *hash_table)
ensure_valid_bucket_pointers(hash_table);
/* Free all the entries. */
- size = ((size_t) 1) << hash_table->size_log2;
+ size = NUM_BUCKETS(hash_table->size_log2);
for (i = 0; i < size; ++i)
{
dsa_pointer item_pointer = hash_table->buckets[i];
@@ -592,6 +600,145 @@ dshash_memhash(const void *v, size_t size, void *arg)
return tag_hash(v, size);
}
+/*
+ * dshash_seq_init/_next/_term
+ * Sequentially scan through dshash table and return all the
+ * elements one by one, return NULL when no more.
+ *
+ * dshash_seq_term should always be called when a scan finished.
+ * The caller may delete returned elements midst of a scan by using
+ * dshash_delete_current(). exclusive must be true to delete elements.
+ */
+void
+dshash_seq_init(dshash_seq_status *status, dshash_table *hash_table,
+ bool exclusive)
+{
+ status->hash_table = hash_table;
+ status->curbucket = 0;
+ status->nbuckets = 0;
+ status->curitem = NULL;
+ status->pnextitem = InvalidDsaPointer;
+ status->curpartition = -1;
+ status->exclusive = exclusive;
+}
+
+/*
+ * Returns the next element.
+ *
+ * Returned elements are locked and the caller must not explicitly release
+ * it. It is released at the next call to dshash_next().
+ */
+void *
+dshash_seq_next(dshash_seq_status *status)
+{
+ dsa_pointer next_item_pointer;
+
+ if (status->curitem == NULL)
+ {
+ int partition;
+
+ Assert(status->curbucket == 0);
+ Assert(!status->hash_table->find_locked);
+
+ /* first shot. grab the first item. */
+ partition =
+ PARTITION_FOR_BUCKET_INDEX(status->curbucket,
+ status->hash_table->size_log2);
+ LWLockAcquire(PARTITION_LOCK(status->hash_table, partition),
+ status->exclusive ? LW_EXCLUSIVE : LW_SHARED);
+ status->curpartition = partition;
+
+ /* resize doesn't happen from now until seq scan ends */
+ status->nbuckets =
+ NUM_BUCKETS(status->hash_table->control->size_log2);
+ ensure_valid_bucket_pointers(status->hash_table);
+
+ next_item_pointer = status->hash_table->buckets[status->curbucket];
+ }
+ else
+ next_item_pointer = status->pnextitem;
+
+ /* Move to the next bucket if we finished the current bucket */
+ while (!DsaPointerIsValid(next_item_pointer))
+ {
+ int next_partition;
+
+ if (++status->curbucket >= status->nbuckets)
+ {
+ /* all buckets have been scanned. finish. */
+ return NULL;
+ }
+
+ /* Check if move to the next partition */
+ next_partition =
+ PARTITION_FOR_BUCKET_INDEX(status->curbucket,
+ status->hash_table->size_log2);
+
+ if (status->curpartition != next_partition)
+ {
+ /*
+ * Move to the next partition. Lock the next partition then release
+ * the current, not in the reverse order to avoid concurrent
+ * resizing. Avoid dead lock by taking lock in the same order
+ * with resize().
+ */
+ LWLockAcquire(PARTITION_LOCK(status->hash_table,
+ next_partition),
+ status->exclusive ? LW_EXCLUSIVE : LW_SHARED);
+ LWLockRelease(PARTITION_LOCK(status->hash_table,
+ status->curpartition));
+ status->curpartition = next_partition;
+ }
+
+ next_item_pointer = status->hash_table->buckets[status->curbucket];
+ }
+
+ status->curitem =
+ dsa_get_address(status->hash_table->area, next_item_pointer);
+ status->hash_table->find_locked = true;
+ status->hash_table->find_exclusively_locked = status->exclusive;
+
+ /*
+ * The caller may delete the item. Store the next item in case of deletion.
+ */
+ status->pnextitem = status->curitem->next;
+
+ return ENTRY_FROM_ITEM(status->curitem);
+}
+
+/*
+ * Terminates the seqscan and release all locks.
+ *
+ * Should be always called when finishing or exiting a seqscan.
+ */
+void
+dshash_seq_term(dshash_seq_status *status)
+{
+ status->hash_table->find_locked = false;
+ status->hash_table->find_exclusively_locked = false;
+
+ if (status->curpartition >= 0)
+ LWLockRelease(PARTITION_LOCK(status->hash_table, status->curpartition));
+}
+
+/* Remove the current entry while a seq scan. */
+void
+dshash_delete_current(dshash_seq_status *status)
+{
+ dshash_table *hash_table = status->hash_table;
+ dshash_table_item *item = status->curitem;
+ size_t partition = PARTITION_FOR_HASH(item->hash);
+
+ Assert(status->exclusive);
+ Assert(hash_table->control->magic == DSHASH_MAGIC);
+ Assert(hash_table->find_locked);
+ Assert(hash_table->find_exclusively_locked);
+ Assert(LWLockHeldByMeInMode(PARTITION_LOCK(hash_table, partition),
+ LW_EXCLUSIVE));
+
+ delete_item(hash_table, item);
+}
+
/*
* Print debugging information about the internal state of the hash table to
* stderr. The caller must hold no partition locks.
diff --git a/src/include/lib/dshash.h b/src/include/lib/dshash.h
index b86df68e77..0ca9514021 100644
--- a/src/include/lib/dshash.h
+++ b/src/include/lib/dshash.h
@@ -59,6 +59,21 @@ typedef struct dshash_parameters
struct dshash_table_item;
typedef struct dshash_table_item dshash_table_item;
+/*
+ * Sequential scan state. The detail is exposed to let users know the storage
+ * size but it should be considered as an opaque type by callers.
+ */
+typedef struct dshash_seq_status
+{
+ dshash_table *hash_table; /* dshash table working on */
+ int curbucket; /* bucket number we are at */
+ int nbuckets; /* total number of buckets in the dshash */
+ dshash_table_item *curitem; /* item we are currently at */
+ dsa_pointer pnextitem; /* dsa-pointer to the next item */
+ int curpartition; /* partition number we are at */
+ bool exclusive; /* locking mode */
+} dshash_seq_status;
+
/* Creating, sharing and destroying from hash tables. */
extern dshash_table *dshash_create(dsa_area *area,
const dshash_parameters *params,
@@ -80,6 +95,12 @@ extern bool dshash_delete_key(dshash_table *hash_table, const void *key);
extern void dshash_delete_entry(dshash_table *hash_table, void *entry);
extern void dshash_release_lock(dshash_table *hash_table, void *entry);
+/* seq scan support */
+extern void dshash_seq_init(dshash_seq_status *status, dshash_table *hash_table,
+ bool exclusive);
+extern void *dshash_seq_next(dshash_seq_status *status);
+extern void dshash_seq_term(dshash_seq_status *status);
+extern void dshash_delete_current(dshash_seq_status *status);
/* Convenience hash and compare functions wrapping memcmp and tag_hash. */
extern int dshash_memcmp(const void *a, const void *b, size_t size, void *arg);
extern dshash_hash dshash_memhash(const void *v, size_t size, void *arg);
--
2.18.4
v36-0003-Add-conditional-lock-feature-to-dshash.patch (text/x-patch, 6.2K) download
From 2d7795df24308e52340613dfe3f5106432ff1fee Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Fri, 13 Mar 2020 16:58:57 +0900
Subject: [PATCH v36 3/7] Add conditional lock feature to dshash
Dshash currently waits for lock unconditionally. It is inconvenient
when we want to avoid being blocked by other processes. This commit
adds alternative functions of dshash_find and dshash_find_or_insert
that allows immediate return on lock failure.
---
src/backend/lib/dshash.c | 98 +++++++++++++++++++++-------------------
src/include/lib/dshash.h | 3 ++
2 files changed, 55 insertions(+), 46 deletions(-)
diff --git a/src/backend/lib/dshash.c b/src/backend/lib/dshash.c
index 1ef093e2e9..d7ee6de11e 100644
--- a/src/backend/lib/dshash.c
+++ b/src/backend/lib/dshash.c
@@ -383,6 +383,10 @@ dshash_get_hash_table_handle(dshash_table *hash_table)
* the caller must take care to ensure that the entry is not left corrupted.
* The lock mode is either shared or exclusive depending on 'exclusive'.
*
+ * If found is not NULL, *found is set to true if the key is found in the hash
+ * table. If the key is not found, *found is set to false and a pointer to a
+ * newly created entry is returned.
+ *
* The caller must not lock a lock already.
*
* Note that the lock held is in fact an LWLock, so interrupts will be held on
@@ -392,36 +396,7 @@ dshash_get_hash_table_handle(dshash_table *hash_table)
void *
dshash_find(dshash_table *hash_table, const void *key, bool exclusive)
{
- dshash_hash hash;
- size_t partition;
- dshash_table_item *item;
-
- hash = hash_key(hash_table, key);
- partition = PARTITION_FOR_HASH(hash);
-
- Assert(hash_table->control->magic == DSHASH_MAGIC);
- Assert(!hash_table->find_locked);
-
- LWLockAcquire(PARTITION_LOCK(hash_table, partition),
- exclusive ? LW_EXCLUSIVE : LW_SHARED);
- ensure_valid_bucket_pointers(hash_table);
-
- /* Search the active bucket. */
- item = find_in_bucket(hash_table, key, BUCKET_FOR_HASH(hash_table, hash));
-
- if (!item)
- {
- /* Not found. */
- LWLockRelease(PARTITION_LOCK(hash_table, partition));
- return NULL;
- }
- else
- {
- /* The caller will free the lock by calling dshash_release_lock. */
- hash_table->find_locked = true;
- hash_table->find_exclusively_locked = exclusive;
- return ENTRY_FROM_ITEM(item);
- }
+ return dshash_find_extended(hash_table, key, exclusive, false, false, NULL);
}
/*
@@ -439,31 +414,60 @@ dshash_find_or_insert(dshash_table *hash_table,
const void *key,
bool *found)
{
- dshash_hash hash;
- size_t partition_index;
- dshash_partition *partition;
+ return dshash_find_extended(hash_table, key, true, false, true, found);
+}
+
+
+/*
+ * Find the key in the hash table.
+ *
+ * "exclusive" is the lock mode in which the partition for the returned item
+ * is locked. If "nowait" is true, the function immediately returns if
+ * required lock was not acquired. "insert" indicates insert mode. In this
+ * mode new entry is inserted and set *found to false. *found is set to true if
+ * found. "found" must be non-null in this mode.
+ */
+void *
+dshash_find_extended(dshash_table *hash_table, const void *key,
+ bool exclusive, bool nowait, bool insert, bool *found)
+{
+ dshash_hash hash = hash_key(hash_table, key);
+ size_t partidx = PARTITION_FOR_HASH(hash);
+ dshash_partition *partition = &hash_table->control->partitions[partidx];
+ LWLockMode lockmode = exclusive ? LW_EXCLUSIVE : LW_SHARED;
dshash_table_item *item;
- hash = hash_key(hash_table, key);
- partition_index = PARTITION_FOR_HASH(hash);
- partition = &hash_table->control->partitions[partition_index];
-
- Assert(hash_table->control->magic == DSHASH_MAGIC);
- Assert(!hash_table->find_locked);
+ /* must be exclusive when insert allowed */
+ Assert(!insert || (exclusive && found != NULL));
restart:
- LWLockAcquire(PARTITION_LOCK(hash_table, partition_index),
- LW_EXCLUSIVE);
+ if (!nowait)
+ LWLockAcquire(PARTITION_LOCK(hash_table, partidx), lockmode);
+ else if (!LWLockConditionalAcquire(PARTITION_LOCK(hash_table, partidx),
+ lockmode))
+ return NULL;
+
ensure_valid_bucket_pointers(hash_table);
/* Search the active bucket. */
item = find_in_bucket(hash_table, key, BUCKET_FOR_HASH(hash_table, hash));
if (item)
- *found = true;
+ {
+ if (found)
+ *found = true;
+ }
else
{
- *found = false;
+ if (found)
+ *found = false;
+
+ if (!insert)
+ {
+ /* The caller didn't told to add a new entry. */
+ LWLockRelease(PARTITION_LOCK(hash_table, partidx));
+ return NULL;
+ }
/* Check if we are getting too full. */
if (partition->count > MAX_COUNT_PER_PARTITION(hash_table))
@@ -479,7 +483,8 @@ restart:
* Give up our existing lock first, because resizing needs to
* reacquire all the locks in the right order to avoid deadlocks.
*/
- LWLockRelease(PARTITION_LOCK(hash_table, partition_index));
+ LWLockRelease(PARTITION_LOCK(hash_table, partidx));
+
resize(hash_table, hash_table->size_log2 + 1);
goto restart;
@@ -493,12 +498,13 @@ restart:
++partition->count;
}
- /* The caller must release the lock with dshash_release_lock. */
+ /* The caller will free the lock by calling dshash_release_lock. */
hash_table->find_locked = true;
- hash_table->find_exclusively_locked = true;
+ hash_table->find_exclusively_locked = exclusive;
return ENTRY_FROM_ITEM(item);
}
+
/*
* Remove an entry by key. Returns true if the key was found and the
* corresponding entry was removed.
diff --git a/src/include/lib/dshash.h b/src/include/lib/dshash.h
index 0ca9514021..5f7a60febd 100644
--- a/src/include/lib/dshash.h
+++ b/src/include/lib/dshash.h
@@ -91,6 +91,9 @@ extern void *dshash_find(dshash_table *hash_table,
const void *key, bool exclusive);
extern void *dshash_find_or_insert(dshash_table *hash_table,
const void *key, bool *found);
+extern void *dshash_find_extended(dshash_table *hash_table, const void *key,
+ bool exclusive, bool nowait, bool insert,
+ bool *found);
extern bool dshash_delete_key(dshash_table *hash_table, const void *key);
extern void dshash_delete_entry(dshash_table *hash_table, void *entry);
extern void dshash_release_lock(dshash_table *hash_table, void *entry);
--
2.18.4
v36-0004-Make-archiver-process-an-auxiliary-process.patch (text/x-patch, 17.6K) download
From 14bd15cad98e30d1e269087874f29465bd401a9f Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Fri, 13 Mar 2020 16:59:38 +0900
Subject: [PATCH v36 4/7] Make archiver process an auxiliary process
This is a preliminary patch for shared-memory based stats collector.
Archiver process must be a auxiliary process since it uses shared
memory after stats data was moved into shared-memory. Make the process
an auxiliary process in order to make it work.
---
src/backend/access/transam/xlogarchive.c | 6 +-
src/backend/bootstrap/bootstrap.c | 22 ++--
src/backend/postmaster/pgarch.c | 130 +++--------------------
src/backend/postmaster/postmaster.c | 50 +++++----
src/backend/storage/lmgr/proc.c | 1 +
src/include/access/xlog.h | 3 +
src/include/access/xlogarchive.h | 1 +
src/include/miscadmin.h | 2 +
src/include/postmaster/pgarch.h | 4 +-
src/include/storage/pmsignal.h | 1 -
src/include/storage/proc.h | 3 +
11 files changed, 69 insertions(+), 154 deletions(-)
diff --git a/src/backend/access/transam/xlogarchive.c b/src/backend/access/transam/xlogarchive.c
index 8f8734dc1d..2e6c322142 100644
--- a/src/backend/access/transam/xlogarchive.c
+++ b/src/backend/access/transam/xlogarchive.c
@@ -29,7 +29,9 @@
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/lwlock.h"
+#include "storage/latch.h"
#include "storage/pmsignal.h"
+#include "storage/proc.h"
/*
* Attempt to retrieve the specified file from off-line archival storage.
@@ -489,8 +491,8 @@ XLogArchiveNotify(const char *xlog)
}
/* Notify archiver that it's got something to do */
- if (IsUnderPostmaster)
- SendPostmasterSignal(PMSIGNAL_WAKEN_ARCHIVER);
+ if (IsUnderPostmaster && ProcGlobal->archiverLatch)
+ SetLatch(ProcGlobal->archiverLatch);
}
/*
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..81bfaea869 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -317,6 +317,9 @@ AuxiliaryProcessMain(int argc, char *argv[])
case StartupProcess:
MyBackendType = B_STARTUP;
break;
+ case ArchiverProcess:
+ MyBackendType = B_ARCHIVER;
+ break;
case BgWriterProcess:
MyBackendType = B_BG_WRITER;
break;
@@ -437,30 +440,29 @@ AuxiliaryProcessMain(int argc, char *argv[])
proc_exit(1); /* should never return */
case StartupProcess:
- /* don't set signals, startup process has its own agenda */
StartupProcessMain();
- proc_exit(1); /* should never return */
+ proc_exit(1);
+
+ case ArchiverProcess:
+ PgArchiverMain();
+ proc_exit(1);
case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
BackgroundWriterMain();
- proc_exit(1); /* should never return */
+ proc_exit(1);
case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
CheckpointerMain();
- proc_exit(1); /* should never return */
+ proc_exit(1);
case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
InitXLOGAccess();
WalWriterMain();
- proc_exit(1); /* should never return */
+ proc_exit(1);
case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
WalReceiverMain();
- proc_exit(1); /* should never return */
+ proc_exit(1);
default:
elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 37be0e2bbb..063d1323ea 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -48,6 +48,7 @@
#include "storage/latch.h"
#include "storage/pg_shmem.h"
#include "storage/pmsignal.h"
+#include "storage/procsignal.h"
#include "utils/guc.h"
#include "utils/ps_status.h"
@@ -78,13 +79,11 @@
* Local data
* ----------
*/
-static time_t last_pgarch_start_time;
static time_t last_sigterm_time = 0;
/*
* Flags set by interrupt handlers for later service in the main loop.
*/
-static volatile sig_atomic_t wakened = false;
static volatile sig_atomic_t ready_to_stop = false;
/* ----------
@@ -95,8 +94,6 @@ static volatile sig_atomic_t ready_to_stop = false;
static pid_t pgarch_forkexec(void);
#endif
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
static void pgarch_MainLoop(void);
static void pgarch_ArchiverCopyLoop(void);
@@ -110,75 +107,6 @@ static void pgarch_archiveDone(char *xlog);
* ------------------------------------------------------------
*/
-/*
- * pgarch_start
- *
- * Called from postmaster at startup or after an existing archiver
- * died. Attempt to fire up a fresh archiver process.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgarch_start(void)
-{
- time_t curtime;
- pid_t pgArchPid;
-
- /*
- * Do nothing if no archiver needed
- */
- if (!XLogArchivingActive())
- return 0;
-
- /*
- * Do nothing if too soon since last archiver start. This is a safety
- * valve to protect against continuous respawn attempts if the archiver is
- * dying immediately at launch. Note that since we will be re-called from
- * the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
- (unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/* ------------------------------------------------------------
* Local functions called by archiver follow
* ------------------------------------------------------------
@@ -212,14 +140,9 @@ pgarch_forkexec(void)
#endif /* EXEC_BACKEND */
-/*
- * PgArchiverMain
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case. However,
- * since we don't use 'em, it hardly matters...
- */
-NON_EXEC_STATIC void
-PgArchiverMain(int argc, char *argv[])
+/* Main entry point for archiver process */
+void
+PgArchiverMain(void)
{
/*
* Ignore all signals usually bound to some action in the postmaster,
@@ -231,33 +154,26 @@ PgArchiverMain(int argc, char *argv[])
pqsignal(SIGQUIT, SignalHandlerForCrashExit);
pqsignal(SIGALRM, SIG_IGN);
pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, pgarch_waken);
+ pqsignal(SIGUSR1, procsignal_sigusr1_handler);
pqsignal(SIGUSR2, pgarch_waken_stop);
+
/* Reset some signals that are accepted by postmaster but not here */
pqsignal(SIGCHLD, SIG_DFL);
+
+ /* Unblock signals (they were blocked when the postmaster forked us) */
PG_SETMASK(&UnBlockSig);
- MyBackendType = B_ARCHIVER;
- init_ps_display(NULL);
+ /*
+ * Advertise our latch that backends can use to wake us up while we're
+ * sleeping.
+ */
+ ProcGlobal->archiverLatch = &MyProc->procLatch;
pgarch_MainLoop();
exit(0);
}
-/* SIGUSR1 signal handler for archiver process */
-static void
-pgarch_waken(SIGNAL_ARGS)
-{
- int save_errno = errno;
-
- /* set flag that there is work to be done */
- wakened = true;
- SetLatch(MyLatch);
-
- errno = save_errno;
-}
-
/* SIGUSR2 signal handler for archiver process */
static void
pgarch_waken_stop(SIGNAL_ARGS)
@@ -282,14 +198,6 @@ pgarch_MainLoop(void)
pg_time_t last_copy_time = 0;
bool time_to_stop;
- /*
- * We run the copy loop immediately upon entry, in case there are
- * unarchived files left over from a previous database run (or maybe the
- * archiver died unexpectedly). After that we wait for a signal or
- * timeout before doing more.
- */
- wakened = true;
-
/*
* There shouldn't be anything for the archiver to do except to wait for a
* signal ... however, the archiver exists to protect our data, so she
@@ -328,12 +236,8 @@ pgarch_MainLoop(void)
}
/* Do what we're here for */
- if (wakened || time_to_stop)
- {
- wakened = false;
- pgarch_ArchiverCopyLoop();
- last_copy_time = time(NULL);
- }
+ pgarch_ArchiverCopyLoop();
+ last_copy_time = time(NULL);
/*
* Sleep until a signal is received, or until a poll is forced by
@@ -354,13 +258,9 @@ pgarch_MainLoop(void)
WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
timeout * 1000L,
WAIT_EVENT_ARCHIVER_MAIN);
- if (rc & WL_TIMEOUT)
- wakened = true;
if (rc & WL_POSTMASTER_DEATH)
time_to_stop = true;
}
- else
- wakened = true;
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 42223c0f61..4c296dd214 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -554,6 +554,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
#define StartupDataBase() StartChildProcess(StartupProcess)
+#define StartArchiver() StartChildProcess(ArchiverProcess)
#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
#define StartWalWriter() StartChildProcess(WalWriterProcess)
@@ -1799,7 +1800,7 @@ ServerLoop(void)
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartArchiver();
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -3051,7 +3052,7 @@ reaper(SIGNAL_ARGS)
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartArchiver();
if (PgStatPID == 0)
PgStatPID = pgstat_start();
@@ -3186,20 +3187,16 @@ reaper(SIGNAL_ARGS)
}
/*
- * Was it the archiver? If so, just try to start a new one; no need
- * to force reset of the rest of the system. (If fail, we'll try
- * again in future cycles of the main loop.). Unless we were waiting
- * for it to shut down; don't restart it in that case, and
- * PostmasterStateMachine() will advance to the next shutdown step.
+ * Was it the archiver? Normal exit can be ignored; we'll start a new
+ * one at the next iteration of the postmaster's main loop, if
+ * necessary. Any other exit condition is treated as a crash.
*/
if (pid == PgArchPID)
{
PgArchPID = 0;
if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("archiver process"),
- pid, exitstatus);
- if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ HandleChildCrash(pid, exitstatus,
+ _("archiver process"));
continue;
}
@@ -3447,7 +3444,7 @@ CleanupBackend(int pid,
/*
* HandleChildCrash -- cleanup after failed backend, bgwriter, checkpointer,
- * walwriter, autovacuum, or background worker.
+ * walwriter, autovacuum, archiver or background worker.
*
* The objectives here are to clean up our local state about the child
* process, and to signal all other remaining children to quickdie.
@@ -3652,6 +3649,18 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(AutoVacPID, (SendStop ? SIGSTOP : SIGQUIT));
}
+ /* Take care of the archiver too */
+ if (pid == PgArchPID)
+ PgArchPID = 0;
+ else if (PgArchPID != 0 && take_action)
+ {
+ ereport(DEBUG2,
+ (errmsg_internal("sending %s to process %d",
+ (SendStop ? "SIGSTOP" : "SIGQUIT"),
+ (int) PgArchPID)));
+ signal_child(PgArchPID, (SendStop ? SIGSTOP : SIGQUIT));
+ }
+
/*
* Force a power-cycle of the pgarch process too. (This isn't absolutely
* necessary, but it seems like a good idea for robustness, and it
@@ -3948,6 +3957,7 @@ PostmasterStateMachine(void)
Assert(CheckpointerPID == 0);
Assert(WalWriterPID == 0);
Assert(AutoVacPID == 0);
+ Assert(PgArchPID == 0);
/* syslogger is not considered here */
pmState = PM_NO_CHILDREN;
}
@@ -5223,7 +5233,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartArchiver();
/*
* If we aren't planning to enter hot standby mode later, treat
@@ -5268,16 +5278,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (StartWorkerNeeded || HaveCrashedWorker)
maybe_start_bgworkers();
- if (CheckPostmasterSignal(PMSIGNAL_WAKEN_ARCHIVER) &&
- PgArchPID != 0)
- {
- /*
- * Send SIGUSR1 to archiver process, to wake it up and begin archiving
- * next WAL file.
- */
- signal_child(PgArchPID, SIGUSR1);
- }
-
/* Tell syslogger to rotate logfile if requested */
if (SysLoggerPID != 0)
{
@@ -5515,6 +5515,10 @@ StartChildProcess(AuxProcType type)
ereport(LOG,
(errmsg("could not fork startup process: %m")));
break;
+ case ArchiverProcess:
+ ereport(LOG,
+ (errmsg("could not fork archiver process: %m")));
+ break;
case BgWriterProcess:
ereport(LOG,
(errmsg("could not fork background writer process: %m")));
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index aa9fbd8054..9cc5226667 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -181,6 +181,7 @@ InitProcGlobal(void)
ProcGlobal->startupBufferPinWaitBufId = -1;
ProcGlobal->walwriterLatch = NULL;
ProcGlobal->checkpointerLatch = NULL;
+ ProcGlobal->archiverLatch = NULL;
pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PGPROCNO);
pg_atomic_init_u32(&ProcGlobal->clogGroupFirst, INVALID_PGPROCNO);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e71..65443063e8 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -339,6 +339,9 @@ extern XLogRecPtr GetRedoRecPtr(void);
extern XLogRecPtr GetInsertRecPtr(void);
extern XLogRecPtr GetFlushRecPtr(void);
extern XLogRecPtr GetLastImportantRecPtr(void);
+extern void XLogArchiveWakeupStart(void);
+extern void XLogArchiveWakeupEnd(void);
+extern void XLogArchiveWakeup(void);
extern void RemovePromoteSignalFiles(void);
extern bool PromoteIsTriggered(void);
diff --git a/src/include/access/xlogarchive.h b/src/include/access/xlogarchive.h
index 1c67de2ede..54ce0b97d7 100644
--- a/src/include/access/xlogarchive.h
+++ b/src/include/access/xlogarchive.h
@@ -25,6 +25,7 @@ extern void ExecuteRecoveryCommand(const char *command, const char *commandName,
extern void KeepFileRestoredFromArchive(const char *path, const char *xlogfname);
extern void XLogArchiveNotify(const char *xlog);
extern void XLogArchiveNotifySeg(XLogSegNo segno);
+extern void XLogArchiveWakeup(void);
extern void XLogArchiveForceDone(const char *xlog);
extern bool XLogArchiveCheckDone(const char *xlog);
extern bool XLogArchiveIsBusy(const char *xlog);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 72e3352398..bba002ad24 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -418,6 +418,7 @@ typedef enum
BootstrapProcess,
StartupProcess,
BgWriterProcess,
+ ArchiverProcess,
CheckpointerProcess,
WalWriterProcess,
WalReceiverProcess,
@@ -430,6 +431,7 @@ extern AuxProcType MyAuxProcType;
#define AmBootstrapProcess() (MyAuxProcType == BootstrapProcess)
#define AmStartupProcess() (MyAuxProcType == StartupProcess)
#define AmBackgroundWriterProcess() (MyAuxProcType == BgWriterProcess)
+#define AmArchiverProcess() (MyAuxProcType == ArchiverProcess)
#define AmCheckpointerProcess() (MyAuxProcType == CheckpointerProcess)
#define AmWalWriterProcess() (MyAuxProcType == WalWriterProcess)
#define AmWalReceiverProcess() (MyAuxProcType == WalReceiverProcess)
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..e3ffc63f14 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -32,8 +32,6 @@
*/
extern int pgarch_start(void);
-#ifdef EXEC_BACKEND
-extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void PgArchiverMain(void) pg_attribute_noreturn();
#endif /* _PGARCH_H */
diff --git a/src/include/storage/pmsignal.h b/src/include/storage/pmsignal.h
index 56c5ec4481..c691acf8cd 100644
--- a/src/include/storage/pmsignal.h
+++ b/src/include/storage/pmsignal.h
@@ -34,7 +34,6 @@ typedef enum
{
PMSIGNAL_RECOVERY_STARTED, /* recovery has started */
PMSIGNAL_BEGIN_HOT_STANDBY, /* begin Hot Standby */
- PMSIGNAL_WAKEN_ARCHIVER, /* send a NOTIFY signal to xlog archiver */
PMSIGNAL_ROTATE_LOGFILE, /* send SIGUSR1 to syslogger to rotate logfile */
PMSIGNAL_START_AUTOVAC_LAUNCHER, /* start an autovacuum launcher */
PMSIGNAL_START_AUTOVAC_WORKER, /* start an autovacuum worker */
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 9c9a50ae45..de20520b8c 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -345,6 +345,9 @@ typedef struct PROC_HDR
int startupProcPid;
/* Buffer id of the buffer that Startup process waits for pin on, or -1 */
int startupBufferPinWaitBufId;
+ /* Archiver process's latch */
+ Latch *archiverLatch;
+ /* Current shared estimate of appropriate spins_per_delay value */
} PROC_HDR;
extern PGDLLIMPORT PROC_HDR *ProcGlobal;
--
2.18.4
v36-0005-Shared-memory-based-stats-collector.patch (text/x-patch, 257.6K) download
From d59c9e2a9a1dba9e4e80e8b443963191c42e8d8c Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Thu, 19 Mar 2020 15:11:14 +0900
Subject: [PATCH v36 5/7] Shared-memory based stats collector
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 500ms and such large data is
serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 500ms. Locks
on the shared statistics is acquired by the units of such like tables,
functions so the expected chance of collision are not so
high. Furthermore, until 1 second has elapsed since the last flushing
to shared stats, lock failure postpones stats flushing so that lock
contention doesn't slow down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/heap/heapam_handler.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/access/transam/xlog.c | 4 +-
src/backend/catalog/index.c | 24 +-
src/backend/commands/analyze.c | 8 +-
src/backend/commands/dbcommands.c | 2 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/vacuum.c | 4 +-
src/backend/postmaster/autovacuum.c | 74 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 24 +-
src/backend/postmaster/interrupt.c | 5 +-
src/backend/postmaster/pgarch.c | 12 +-
src/backend/postmaster/pgstat.c | 5128 +++++++----------
src/backend/postmaster/postmaster.c | 88 +-
src/backend/replication/basebackup.c | 4 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 4 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/smgr.c | 4 +-
src/backend/tcop/postgres.c | 37 +-
src/backend/utils/adt/pgstatfuncs.c | 59 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 3 -
src/backend/utils/init/postinit.c | 11 +
src/backend/utils/misc/guc.c | 14 +-
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/include/miscadmin.h | 3 +-
src/include/pgstat.h | 581 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/guc_tables.h | 2 +-
src/include/utils/timeout.h | 1 +
34 files changed, 2256 insertions(+), 3875 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index dcaea7135f..49df584a9e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1061,8 +1061,8 @@ heapam_scan_analyze_next_tuple(TableScanDesc scan, TransactionId OldestXmin,
* our own. In this case we should count and sample the row,
* to accommodate users who load a table and analyze it in one
* transaction. (pgstat_report_analyze has to adjust the
- * numbers we send to the stats collector to make this come
- * out right.)
+ * numbers we report to the activity stats facility to make
+ * this come out right.)
*/
if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(targtuple->t_data)))
{
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 92389e6666..82f7bbcca7 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -599,7 +599,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
new_min_multi,
false);
- /* report results to the stats collector, too */
+ /* report results to the activity stats facility, too */
pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
Max(new_live_tuples, 0),
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 09c01ed4ae..4b10efee81 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -8576,9 +8576,9 @@ LogCheckpointEnd(bool restartpoint)
&sync_secs, &sync_usecs);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time +=
+ BgWriterStats.checkpoint_write_time +=
write_secs * 1000 + write_usecs / 1000;
- BgWriterStats.m_checkpoint_sync_time +=
+ BgWriterStats.checkpoint_sync_time +=
sync_secs * 1000 + sync_usecs / 1000;
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 117e3fdef7..622083b2c0 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1688,28 +1688,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be flushed by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Close relations */
table_close(pg_class, RowExclusiveLock);
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 8af12b5c6b..a7e787d9d1 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -644,10 +644,10 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
}
/*
- * Report ANALYZE to the stats collector, too. However, if doing
- * inherited stats we shouldn't report, because the stats collector only
- * tracks per-table stats. Reset the changes_since_analyze counter only
- * if we analyzed all columns; otherwise, there is still work for
+ * Report ANALYZE to the activity stats facility, too. However, if doing
+ * inherited stats we shouldn't report, because the activity stats facility
+ * only tracks per-table stats. Reset the changes_since_analyze counter
+ * only if we analyzed all columns; otherwise, there is still work for
* auto-analyze to do.
*/
if (!inh)
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index f27c3fe8c1..de0c749570 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -971,7 +971,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
DropDatabaseBuffers(db_id);
/*
- * Tell the stats collector to forget it immediately, too.
+ * Tell the active stats facility to forget it immediately, too.
*/
pgstat_drop_database(db_id);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index f80a9e96a9..e7ccc6eba7 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -336,10 +336,10 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
/*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
+ * Inform activity stats facility about our activity: basically, we
+ * truncated the matview and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
*/
pgstat_count_truncate(matviewRel);
if (!stmt->skipData)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ddeec870d8..c5477ff567 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -319,8 +319,8 @@ vacuum(List *relations, VacuumParams *params,
errmsg("VACUUM option DISABLE_PAGE_SKIPPING cannot be used with FULL")));
/*
- * Send info about dead objects to the statistics collector, unless we are
- * in autovacuum --- autovacuum.c does this for itself.
+ * Send info about dead objects to the activity statistics facility, unless
+ * we are in autovacuum --- autovacuum.c does this for itself.
*/
if ((params->options & VACOPT_VACUUM) && !IsAutoVacuumWorkerProcess())
pgstat_vacuum_stat();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 1b8cd7bacd..5abb96d2ba 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -338,9 +338,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1664,12 +1661,12 @@ AutoVacWorkerMain(int argc, char *argv[])
char dbname[NAMEDATALEN];
/*
- * Report autovac startup to the stats collector. We deliberately do
- * this before InitPostgres, so that the last_autovac_time will get
- * updated even if the connection attempt fails. This is to prevent
- * autovac from getting "stuck" repeatedly selecting an unopenable
- * database, rather than making any progress on stuff it can connect
- * to.
+ * Report autovac startup to the activity stats facility. We
+ * deliberately do this before InitPostgres, so that the
+ * last_autovac_time will get updated even if the connection attempt
+ * fails. This is to prevent autovac from getting "stuck" repeatedly
+ * selecting an unopenable database, rather than making any progress on
+ * stuff it can connect to.
*/
pgstat_report_autovac(dbid);
@@ -1941,8 +1938,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1961,17 +1956,11 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
/*
- * Clean up any dead statistics collector entries for this DB. We always
+ * Clean up any dead activity statistics entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
* nothing worth vacuuming in the database.
*/
@@ -2014,9 +2003,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2095,8 +2081,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2179,8 +2165,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2739,29 +2725,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2782,17 +2745,12 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
bool doanalyze;
autovac_table *tab = NULL;
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
bool wraparound;
AutoVacOpts *avopts;
/* use fresh stats */
autovac_refresh_stats();
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* fetch the relation's relcache entry */
classTup = SearchSysCacheCopy1(RELOID, ObjectIdGetDatum(relid));
if (!HeapTupleIsValid(classTup))
@@ -2816,8 +2774,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
}
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2939,7 +2897,7 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
*
* For analyze, the analysis done is that the number of tuples inserted,
* deleted and updated since the last analyze exceeds a threshold calculated
- * in the same fashion as above. Note that the collector actually stores
+ * in the same fashion as above. Note that the activity statistics stores
* the number of tuples (both live and dead) that there were as of the last
* analyze. This is asymmetric to the VACUUM case.
*
@@ -2949,8 +2907,8 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
*
* A table whose autovacuum_enabled option is false is
* automatically skipped (unless we have to vacuum it due to freeze_max_age).
- * Thus autovacuum can be disabled for specific tables. Also, when the stats
- * collector does not have data about a table, it will be skipped.
+ * Thus autovacuum can be disabled for specific tables. Also, when the activity
+ * statistics does not have data about a table, it will be skipped.
*
* A table whose vac_base_thresh value is < 0 takes the base value from the
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..4382b1726f 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -234,9 +234,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Send off activity statistics to the activity stats facility
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 624a3238b8..dd36b4965e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -350,7 +350,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ BgWriterStats.requested_checkpoints++;
}
/*
@@ -364,7 +364,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ BgWriterStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -486,13 +486,13 @@ CheckpointerMain(void)
CheckArchiveTimeout();
/*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
+ * Send off activity statistics to the activity stats facility. (The
+ * reason why we re-use bgwriter-related code for this is that the
+ * bgwriter and checkpointer used to be just one process. It's
+ * probably not worth the trouble to split the stats support into two
+ * independent stats message types.)
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* If any checkpoint flags have been set, redo the loop to handle the
@@ -698,9 +698,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Report interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1245,8 +1245,8 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ BgWriterStats.buf_written_backend += CheckpointerShmem->num_backend_writes;
+ BgWriterStats.buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index 3d02439b79..53844eb8bb 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -93,9 +93,8 @@ SignalHandlerForCrashExit(SIGNAL_ARGS)
* shut down and exit.
*
* Typically, this handler would be used for SIGTERM, but some procesess use
- * other signals. In particular, the checkpointer exits on SIGUSR2, the
- * stats collector on SIGQUIT, and the WAL writer exits on either SIGINT
- * or SIGTERM.
+ * other signals. In particular, the checkpointer exits on SIGUSR2, and the WAL
+ * writer exits on either SIGINT or SIGTERM.
*
* ShutdownRequestPending should be checked at a convenient place within the
* main loop, or else the main loop should call HandleMainLoopInterrupts.
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 063d1323ea..08fe87341c 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -372,20 +372,20 @@ pgarch_ArchiverCopyLoop(void)
pgarch_archiveDone(xlog);
/*
- * Tell the collector about the WAL file that we successfully
- * archived
+ * Tell the activity statistics facility about the WAL file
+ * that we successfully archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
else
{
/*
- * Tell the collector about the WAL file that we failed to
- * archive
+ * Tell the activity statistics facility about the WAL file
+ * that we failed to archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 5f4b168fd1..90c4ea31a0 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (1000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (250ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (10000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,32 +35,23 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
#include "utils/timestamp.h"
@@ -73,35 +59,20 @@
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 1000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 250 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 10000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -116,7 +87,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -131,15 +101,24 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
-/*
- * BgWriter global statistics counters (unused in other processes).
- * Stored directly in a stats message structure so it can be sent
- * without needing to copy things around. We assume this inits to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ dsa_pointer global_stats; /* DSA pointer to global stats */
+ dsa_pointer archiver_stats; /* Ditto for archiver stats */
+ dsa_pointer slru_stats; /* Ditto for SLRU stats */
+ int refcount; /* # of processes that is attaching the shared
+ * stats memory */
+} StatsShmemStruct;
+
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
/*
* List of SLRU names that we keep stats for. There is no central registry of
@@ -160,72 +139,159 @@ static const char *const slru_names[] = {
#define SLRU_NUM_ELEMENTS lengthof(slru_names)
/*
- * SLRU statistics counts waiting to be sent to the collector. These are
- * stored directly in stats message format so they can be sent without needing
- * to copy things around. We assume this variable inits to zeroes. Entries
- * are one-to-one with slru_names[].
+ * SLRU statistics counts waiting to be written to the shared activity
+ * statistics. We assume this variable inits to zeroes. Entries are
+ * one-to-one with slru_names[].
+ * Changes of SLRU counters are reported within critical sections so we use
+ * static memory in order to avoid memory allocation.
*/
-static PgStat_MsgSLRU SLRUStats[SLRU_NUM_ELEMENTS];
+static PgStat_StatSLRUEntry local_SLRUStats[SLRU_NUM_ELEMENTS];
+static bool have_slrustats = false;
/* ----------
* Local data
* ----------
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
-
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
+ * Enums and types to define shared statistics structure.
+ *
+ * Statistics entries for each object is stored in individual DSA-allocated
+ * memory. Every entry is pointed from the dshash pgStatSharedHash via
+ * dsa_pointer. The structure makes object-stats entries not moved by dshash
+ * resizing, and allows the dshash can release lock sooner on stats
+ * updates. Also it reduces interfering among write-locks on each stat entry by
+ * not relying on partition lock of dshash. PgStatLocalHashEntry is the
+ * local-stats equivalent of PgStatHashEntry for shared stat entries.
+ *
+ * Each stat entry is enveloped by the type PgStatEnvelope, which stores common
+ * attribute of all kind of statistics and a LWLock lock object.
+ *
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStatEnvelope (dsa memory block)
*
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Local stats are stored as:
+ *
+ * dshash pgStatLocalHash
+ * -> PgStatLocalHashEntry (dynahash entry)
+ * (direct pointer)-> PgStatEnvelope (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries. */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_ALL, /* Not a type, for the parameters of
+ * pgstat_collect_stat_entries */
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION /* per-function statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static size_t pgstat_entsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry) /* PGSTAT_TYPE_FUNCTION */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static size_t pgstat_localentsize[] =
+{
+ 0, /* PGSTAT_TYPE_ALL: not an entry */
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry) /* PGSTAT_TYPE_FUNCTION */
+};
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashEntryKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object OID */
+} PgStatHashEntryKey;
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * Stats numbers that are waiting for flushing out to shared stats are held in
+ * pgStatLocalHash,
*/
-static HTAB *pgStatFunctions = NULL;
+typedef struct PgStatHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ dsa_pointer env; /* pointer to shared stats envelope in DSA */
+} PgStatHashEntry;
+
+/* struct for shared statistics entry pointed from shared hash entry. */
+typedef struct PgStatEnvelope
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* databaseid */
+ Oid objectid; /* objectid */
+ size_t len; /* length of body, fixed per type. */
+ LWLock lock; /* lightweight lock to protect body */
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatEnvelope;
+
+#define PgStatEnvelopeSize(bodylen) \
+ (offsetof(PgStatEnvelope, body) + (bodylen))
+
+/* struct for shared SLRU stats */
+typedef struct PgStatSharedSLRUStats
+{
+ PgStat_StatSLRUEntry entry[SLRU_NUM_ELEMENTS];
+ LWLock lock;
+} PgStatSharedSLRUStats;
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashEntryKey key; /* hash key */
+ PgStatEnvelope *env; /* pointer to stats envelope in heap */
+} PgStatLocalHashEntry;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * Snapshot is stats entry that is locally copied to offset stable values for a
+ * transaction.
*/
-static bool have_function_stats = false;
+typedef struct PgStatSnapshot
+{
+ PgStatHashEntryKey key;
+ bool negative;
+ int body[FLEXIBLE_ARRAY_MEMBER]; /* statistics body */
+} PgStatSnapshot;
+
+#define PgStatSnapshotSize(bodylen) \
+ (offsetof(PgStatSnapshot, body) + (bodylen))
+
+/* parameter for shared hashes */
+static const dshash_parameters dsh_rootparams = {
+ sizeof(PgStatHashEntryKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
+
+/* Local stats numbers are stored here */
+static HTAB *pgStatLocalHash = NULL;
+
+#define HAVE_ANY_PENDING_STATS() \
+ (pgStatLocalHash != NULL || \
+ pgStatXactCommit != 0 || pgStatXactRollback != 0)
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -262,11 +328,10 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
+static MemoryContext pgStatSnapshotContext = NULL;
+static HTAB *pgStatSnapshotHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -275,20 +340,19 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Cluster wide statistics.
+ *
+ * Contains statistics that are collected not per database nor per table
+ * basis. shared_* points to shared memory and snapshot_* are backend
+ * snapshots.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+static bool global_snapshot_is_valid = false;
+static PgStat_ArchiverStats *shared_archiverStats;
+static PgStat_ArchiverStats snapshot_archiverStats;
+static PgStat_GlobalStats *shared_globalStats;
+static PgStat_GlobalStats snapshot_globalStats;
+static PgStatSharedSLRUStats *shared_SLRUStats;
+static PgStat_StatSLRUEntry snapshot_SLRUStats[SLRU_NUM_ELEMENTS];
/*
* Total time charged to functions so far in the current backend.
@@ -297,526 +361,278 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/*
+ * Newly created shared stats entries needs to be initialized before the other
+ * processes get access it. get_stat_entry() calls it for the purpose.
+ */
+typedef void (*entry_initializer) (PgStatEnvelope * env);
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStatEnvelope * get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait,
+ entry_initializer initfunc, bool *found);
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid);
+static void create_missing_dbentries(void);
+static void pgstat_write_database_stats(PgStat_StatDBEntry *dbentry);
+static void pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry);
+
+static bool flush_tabstat(PgStatEnvelope * env, bool nowait);
+static bool flush_funcstat(PgStatEnvelope * env, bool nowait);
+static bool flush_dbstat(PgStatEnvelope * env, bool nowait);
+static bool flush_slrustat(bool nowait);
+
+static void init_tabentry(PgStatEnvelope * env);
+static void init_funcentry(PgStatEnvelope * env);
+static void init_dbentry(PgStatEnvelope * env);
+
+static bool delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait);
+static PgStatEnvelope * get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static PgStat_SubXactStatus *get_tabstat_stack_level(int nest_level);
+static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level);
+static void pgstat_snapshot_global_stats(void);
+
static void pgstat_read_current_status(void);
-
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
-
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static void pgstat_send_slru(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
-
-static void pgstat_setup_memcxt(void);
-
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_slru(PgStat_MsgSLRU *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
+ bool found;
-#define TESTBYTEVAL ((char) 199)
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(),
+ &found);
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
-
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
- {
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
- }
-
- /*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
- */
- for (addr = addrs; addr; addr = addr->ai_next)
+ if (!IsUnderPostmaster)
{
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
-
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
-
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+ Assert(!found);
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ }
+}
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+/* ----------
+ * pgstat_setup_memcxt() -
+ *
+ * Create pgStatLocalContext and pgStatSnapshotContext, if not already done.
+ * ----------
+ */
+static void
+pgstat_setup_memcxt(void)
+{
+ if (!pgStatLocalContext)
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+
+ if (!pgStatSnapshotContext)
+ pgStatSnapshotContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Database statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+}
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read saved statistics files if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
- test_byte++; /* just make sure variable is changed */
+ /*
+ * Don't use dsm under postmaster, or when not tracking counts.
+ */
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ pgstat_setup_memcxt();
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ if (area)
+ return;
- /* If we get here, we have a working socket */
- break;
- }
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that processes going to exit can find whether it
+ * is the last or not.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
+
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_rootparams, 0);
+
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->global_stats =
+ dsa_allocate0(area, sizeof(PgStat_GlobalStats));
+ StatsShmem->archiver_stats =
+ dsa_allocate0(area, sizeof(PgStat_ArchiverStats));
+ StatsShmem->slru_stats =
+ dsa_allocate0(area, sizeof(PgStatSharedSLRUStats));
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
+
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
+
+ shared_SLRUStats = (PgStatSharedSLRUStats *)
+ dsa_get_address(area, StatsShmem->slru_stats);
+ LWLockInitialize(&shared_SLRUStats->lock, LWTRANCHE_STATS);
+
+ /* Load saved data if any. */
+ pgstat_read_statsfiles();
+
+ StatsShmem->refcount = 1;
}
+ LWLockRelease(StatsLock);
+
/*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
+ * If we're not the first process, attach existing shared stats area
+ * outside the StatsLock section.
*/
+ if (!area)
{
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- elog(LOG, "getsockopt(SO_RCVBUF) failed: %m");
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- elog(LOG, "setsockopt(SO_RCVBUF) failed: %m");
- }
+ /* Attach shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_rootparams,
+ StatsShmem->hash_handle, 0);
+
+ /* Setup local variables */
+ pgStatLocalHash = NULL;
+ shared_globalStats = (PgStat_GlobalStats *)
+ dsa_get_address(area, StatsShmem->global_stats);
+ shared_archiverStats = (PgStat_ArchiverStats *)
+ dsa_get_address(area, StatsShmem->archiver_stats);
+ shared_SLRUStats = (PgStatSharedSLRUStats *)
+ dsa_get_address(area, StatsShmem->slru_stats);
}
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
-
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
-
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
-
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
-
- /*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
- */
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+ global_snapshot_is_valid = false;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_stats)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
- {
- int nchars;
- Oid tmp_oid;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount < 1)
+ {
/*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
+ * The process is the last one that is attaching the shared stats
+ * memory. Write out the stats files if requested.
*/
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
+ if (write_stats)
+ pgstat_write_statsfiles();
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ /* No one is using the area. */
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
}
- FreeDir(dir);
+
+ LWLockRelease(StatsLock);
+
+ /*
+ * Detach the area. Automatically destroyed when the last process detached
+ * it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+ shared_globalStats = NULL;
+ shared_archiverStats = NULL;
+ pgStatLocalHash = NULL;
+ global_snapshot_is_valid = false;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
-}
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
-#ifdef EXEC_BACKEND
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
- */
-int
-pgstat_start(void)
-{
- time_t curtime;
- pid_t pgStatPid;
-
- /*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
- */
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
-
- /*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
- */
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
/*
- * Okay, fork off the collector.
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
-void
-allow_immediate_pgstat_restart(void)
-{
- last_pgstat_start_time = 0;
+ detach_shared_stats(false); /* Don't write files. */
+ attach_shared_stats();
}
/* ------------------------------------------------------------
@@ -824,147 +640,491 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait = !force; /* Don't use force ever after */
+ HASH_SEQ_STATUS scan;
+ PgStatLocalHashEntry *lent;
+ PgStatLocalHashEntry **dbentlist;
+ int dbentlistlen = 8;
+ int ndbentries = 0;
+ int remains = 0;
int i;
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (area == NULL || !HAVE_ANY_PENDING_STATS())
+ return 0;
+
+ dbentlist = palloc(sizeof(PgStatLocalHashEntry *) * dbentlistlen);
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
+ if (nowait)
+ {
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
+ {
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ nowait = false;
+ }
/*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
+ * flush_tabstat applies some of stats numbers of flushed entries into
+ * local database stats. So flush-out database stats later.
*/
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+ if (pgStatLocalHash)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /* Step 1: flush out other than database stats */
+ hash_seq_init(&scan, pgStatLocalHash);
+ while ((lent = (PgStatLocalHashEntry *) hash_seq_search(&scan)) != NULL)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
+ bool remove = false;
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
+ switch (lent->env->type)
+ {
+ case PGSTAT_TYPE_DB:
+ if (ndbentries >= dbentlistlen)
+ {
+ dbentlistlen *= 2;
+ dbentlist = repalloc(dbentlist,
+ sizeof(PgStatLocalHashEntry *) *
+ dbentlistlen);
+ }
+ dbentlist[ndbentries++] = lent;
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent->env, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent->env, nowait))
+ remove = true;
+ break;
+ default:
+ Assert(false);
+ }
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
+ }
+
+ /* Step 2: flush out database stats */
+ for (i = 0; i < ndbentries; i++)
+ {
+ PgStatLocalHashEntry *lent = dbentlist[i];
+
+ if (flush_dbstat(lent->env, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->env);
+ hash_search(pgStatLocalHash, &lent->key, HASH_REMOVE, NULL);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ pfree(dbentlist);
+
+ if (remains <= 0)
+ {
+ hash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
}
+ /* flush SLRU stats */
+ flush_slrustat(nowait);
+
+ /* Publish the last flush time */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (shared_globalStats->stats_timestamp < now)
+ shared_globalStats->stats_timestamp = now;
+ LWLockRelease(StatsLock);
+
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * If we have pending local stats, let the caller know the retry interval.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
+ if (HAVE_ANY_PENDING_STATS())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
+
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
- /* Finally send SLRU statistics */
- pgstat_send_slru();
+ return retry_interval / 1000;
+ }
+
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
}
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_tabstat(PgStatEnvelope * lenv, bool nowait)
+{
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+ bool found;
+
+ Assert(lenv->type == PGSTAT_TYPE_TABLE);
+
+ lstats = (PgStat_TableStatus *) &lenv->body;
+ dboid = lstats->t_shared ? InvalidOid : MyDatabaseId;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_TABLE, dboid, lstats->t_id,
+ nowait, init_tabentry, &found);
+
+ /* skip if dshash failed to acquire lock */
+ if (shenv == NULL)
+ return false;
+
+ /* retrieve the shared table stats entry from the envelope */
+ shtabstats = (PgStat_StatTabEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated ||
+ lstats->t_counts.vacuum_count > 0 ||
+ lstats->t_counts.analyze_count > 0 ||
+ lstats->t_counts.autovac_vacuum_count > 0 ||
+ lstats->t_counts.autovac_analyze_count > 0)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ /* clear the change counter if requested */
+ if (lstats->t_counts.reset_changed_tuples)
+ shtabstats->changes_since_analyze = 0;
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /*
+ * Update vacuum/analyze timestamp and counters, so that the values won't
+ * goes back.
+ */
+ if (shtabstats->vacuum_timestamp < lstats->vacuum_timestamp)
+ shtabstats->vacuum_timestamp = lstats->vacuum_timestamp;
+ shtabstats->vacuum_count += lstats->t_counts.vacuum_count;
+
+ if (shtabstats->autovac_vacuum_timestamp < lstats->autovac_vacuum_timestamp)
+ shtabstats->autovac_vacuum_timestamp = lstats->autovac_vacuum_timestamp;
+ shtabstats->autovac_vacuum_count += lstats->t_counts.autovac_vacuum_count;
+
+ if (shtabstats->analyze_timestamp < lstats->analyze_timestamp)
+ shtabstats->analyze_timestamp = lstats->analyze_timestamp;
+ shtabstats->analyze_count += lstats->t_counts.analyze_count;
+
+ if (shtabstats->autovac_analyze_timestamp < lstats->autovac_analyze_timestamp)
+ shtabstats->autovac_analyze_timestamp = lstats->autovac_analyze_timestamp;
+ shtabstats->autovac_analyze_count += lstats->t_counts.autovac_analyze_count;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shenv->lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ return true;
+}
+
+/* ----------
+ * init_tabentry() -
+ *
+ * initializes table stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
*/
static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+init_tabentry(PgStatEnvelope * env)
{
- int n;
- int len;
+ PgStat_StatTabEntry *tabent = (PgStat_StatTabEntry *) &env->body;
+
+ /*
+ * If it's a new table entry, initialize counters to the values we just
+ * got.
+ */
+ Assert(env->type == PGSTAT_TYPE_TABLE);
+ tabent->tableid = env->objectid;
+ tabent->numscans = 0;
+ tabent->tuples_returned = 0;
+ tabent->tuples_fetched = 0;
+ tabent->tuples_inserted = 0;
+ tabent->tuples_updated = 0;
+ tabent->tuples_deleted = 0;
+ tabent->tuples_hot_updated = 0;
+ tabent->n_live_tuples = 0;
+ tabent->n_dead_tuples = 0;
+ tabent->changes_since_analyze = 0;
+ tabent->blocks_fetched = 0;
+ tabent->blocks_hit = 0;
+
+ tabent->vacuum_timestamp = 0;
+ tabent->vacuum_count = 0;
+ tabent->autovac_vacuum_timestamp = 0;
+ tabent->autovac_vacuum_count = 0;
+ tabent->analyze_timestamp = 0;
+ tabent->analyze_count = 0;
+ tabent->autovac_analyze_timestamp = 0;
+ tabent->autovac_analyze_count = 0;
+}
+
+
+/*
+ * flush_funcstat - flush out a local function stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatEnvelope * env, bool nowait)
+{
+ /* we assume this inits to all zeroes: */
+ static const PgStat_FunctionCounts all_zeroes;
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStatEnvelope *shenv; /* shared stats envelope */
+ PgStat_StatFuncEntry *sharedent = NULL; /* shared stats entry */
+ bool found;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) &env->body;
+
+ /* Skip it if no counts accumulated for it so far */
+ if (memcmp(&localent->f_counts, &all_zeroes,
+ sizeof(PgStat_FunctionCounts)) == 0)
+ return true;
+
+ /* find shared table stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, localent->f_id,
+ nowait, init_funcentry, &found);
+ /* skip if dshash failed to acquire lock */
+ if (shenv == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* retrieve the shared table stats entry from the envelope */
+ sharedent = (PgStat_StatFuncEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->f_numcalls += localent->f_counts.f_numcalls;
+ sharedent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ sharedent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shenv->lock);
+
+ return true;
+}
+
+
+/* ----------
+ * init_funcentry() -
+ *
+ * initializes function stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
+ */
+static void
+init_funcentry(PgStatEnvelope * env)
+{
+ PgStat_StatFuncEntry *shstat = (PgStat_StatFuncEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ shstat->functionid = env->objectid;
+ shstat->f_numcalls = 0;
+ shstat->f_total_time = 0;
+ shstat->f_self_time = 0;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatEnvelope * env, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStatEnvelope *shenv;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &env->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ shenv = get_stat_entry(PGSTAT_TYPE_DB, localent->databaseid, InvalidOid,
+ nowait, init_dbentry, NULL);
+
+ /* skip if dshash failed to acquire lock */
+ if (!shenv)
+ return false;
+
+ /* retrieve the shared stats entry from the envelope */
+ sharedent = (PgStat_StatDBEntry *) &shenv->body;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shenv->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shenv->lock, LW_EXCLUSIVE))
+ return false;
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(localent->databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -972,257 +1132,149 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&shenv->lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
-/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
+
+/* ----------
+ * init_dbentry() -
+ *
+ * initializes database stats entry
+ * This is also used as initialization callback for get_stat_entry.
+ * ----------
*/
static void
-pgstat_send_funcstats(void)
+init_dbentry(PgStatEnvelope * env)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ Assert(env->type == PGSTAT_TYPE_DB);
+ dbentry->databaseid = env->databaseid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ /* initialize the new shared entry */
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
+}
+
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
+/*
+ * flush_slrustat - flush out a local SLRU stats entries
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if all local SLRU stats are successfully flushed out.
+ */
+static bool
+flush_slrustat(bool nowait)
+{
+ int i;
- if (pgStatFunctions == NULL)
- return;
+ if (!have_slrustats)
+ return true;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&shared_SLRUStats->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&shared_SLRUStats->lock, LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS ; i++)
{
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
+ PgStat_StatSLRUEntry *sharedent = &shared_SLRUStats->entry[i];
+ PgStat_StatSLRUEntry *localent = &local_SLRUStats[i];
+
+ sharedent->blocks_zeroed += localent->blocks_zeroed;
+ sharedent->blocks_hit += localent->blocks_hit;
+ sharedent->blocks_read += localent->blocks_read;
+ sharedent->blocks_written += localent->blocks_written;
+ sharedent->blocks_exists += localent->blocks_exists;
+ sharedent->flush += localent->flush;
+ sharedent->truncate += localent->truncate;
+ }
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
+ /* done, clear the local entry */
+ MemSet(local_SLRUStats, 0,
+ sizeof(PgStat_StatSLRUEntry) * SLRU_NUM_ELEMENTS);
+ LWLockRelease(&shared_SLRUStats->lock);
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
+ have_slrustats = false;
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
+ return true;
+}
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- have_function_stats = false;
+/*
+ * Create the filename for a DB stat file; filename is output parameter points
+ * to a character buffer of length len.
+ */
+static void
+get_dbstat_filename(bool tempname, Oid databaseid, char *filename, int len)
+{
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/db_%u.%s",
+ PGSTAT_STAT_PERMANENT_DIRECTORY,
+ databaseid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
}
/* ----------
- * pgstat_vacuum_stat() -
+ * collect_stat_entries() -
*
- * Will tell the collector about objects he can get rid of.
+ * Collect the shared statistics entries specified by type and dbid. Returns a
+ * list of pointer to shared statistics in palloc'ed memory. If type is
+ * PGSTAT_TYPE_ALL, all types of statistics of the database is collected. If
+ * type is PGSTAT_TYPE_DB, the parameter dbid is ignored and collect all
+ * PGSTAT_TYPE_DB entries.
* ----------
*/
-void
-pgstat_vacuum_stat(void)
+static PgStatEnvelope * *collect_stat_entries(PgStatTypes type, Oid dbid)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
- return;
-
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
-
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
-
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ int listlen = 16;
+ PgStatEnvelope **envlist = palloc(sizeof(PgStatEnvelope * *) * listlen);
+ int n = 0;
+
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ if ((type != PGSTAT_TYPE_ALL && p->key.type != type) ||
+ (type != PGSTAT_TYPE_DB && p->key.databaseid != dbid))
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
+ if (n >= listlen - 1)
{
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
+ listlen *= 2;
+ envlist = repalloc(envlist, listlen * sizeof(PgStatEnvelope * *));
}
+ envlist[n++] = dsa_get_address(area, p->env);
}
+ dshash_seq_term(&hstat);
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
+ envlist[n] = NULL;
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
- continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
-
- hash_destroy(htab);
- }
+ return envlist;
}
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
* into a temporary hash table. Caller should hash_destroy the result
@@ -1231,7 +1283,7 @@ pgstat_vacuum_stat(void)
* ----------
*/
static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
HTAB *htab;
HASHCTL hash_ctl;
@@ -1245,7 +1297,7 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
hash_ctl.entrysize = sizeof(Oid);
hash_ctl.hcxt = CurrentMemoryContext;
htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
+ PGSTAT_TABLE_HASH_SIZE,
&hash_ctl,
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
@@ -1272,65 +1324,185 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
}
+/* ----------
+ * pgstat_vacuum_stat() -
+ *
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
+ * ----------
+ */
+void
+pgstat_vacuum_stat(void)
+{
+ HTAB *dbids; /* database ids */
+ HTAB *relids; /* relation ids in the current database */
+ HTAB *funcids; /* function ids in the current database */
+ PgStatEnvelope **victims; /* victim entry list */
+ int arraylen = 0; /* storage size of the above */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+ int i;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
+
+ /* collect victims from shared stats */
+ arraylen = 16;
+ victims = palloc(sizeof(PgStatEnvelope * *) * arraylen);
+ nvictims = 0;
+
+ dshash_seq_init(&dshstat, pgStatSharedHash, false);
+
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
+ {
+ HTAB *oidtab;
+ Oid *key;
+
+ CHECK_FOR_INTERRUPTS();
+
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
+ continue;
+
+ switch (ent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidtab = dbids;
+ key = &ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidtab = relids;
+ key = &ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidtab = funcids;
+ key = &ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_ALL:
+ Assert(false);
+ break;
+ }
+
+ /* Skip existent objects. */
+ if (hash_search(oidtab, key, HASH_FIND, NULL) != NULL)
+ continue;
+
+ /* extend the list if needed */
+ if (nvictims >= arraylen)
+ {
+ arraylen *= 2;
+ victims = repalloc(victims, sizeof(PgStatEnvelope * *) * arraylen);
+ }
+
+ victims[nvictims++] = dsa_get_address(area, ent->env);
+ }
+ dshash_seq_term(&dshstat);
+ hash_destroy(dbids);
+ hash_destroy(relids);
+ hash_destroy(funcids);
+
+ /* Now try removing the victim entries */
+ for (i = 0; i < nvictims; i++)
+ {
+ PgStatEnvelope *p = victims[i];
+
+ delete_stat_entry(p->type, p->databaseid, p->objectid, true);
+ }
+}
+
+
+/* ----------
+ * delete_stat_entry() -
+ *
+ * Deletes the specified entry from shared stats hash.
+ *
+ * Returns true when successfully deleted.
+ * ----------
+ */
+static bool
+delete_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait)
+{
+ PgStatHashEntryKey key;
+ PgStatHashEntry *ent;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ ent = dshash_find_extended(pgStatSharedHash, &key,
+ true, nowait, false, NULL);
+
+ if (!ent)
+ return false; /* lock failed or not found */
+
+ /* The entry is exclusively locked, so we can free the chunk first. */
+ dsa_free(area, ent->env);
+ dshash_delete_entry(pgStatSharedHash, ent);
+
+ return true;
+}
+
+
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
+ Assert(OidIsValid(databaseid));
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
+ for (p = envlist; *p != NULL; p++)
+ delete_stat_entry((*p)->type, (*p)->databaseid, (*p)->objectid, true);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ pfree(envlist);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1339,20 +1511,47 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **p;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ /* Lookup the entries of the current database in the stats hash. */
+ envlist = collect_stat_entries(PGSTAT_TYPE_ALL, MyDatabaseId);
+ for (p = envlist; *p != NULL; p++)
+ {
+ PgStatEnvelope *env = *p;
+ PgStat_StatDBEntry *dbstat;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+
+ switch (env->type)
+ {
+ case PGSTAT_TYPE_TABLE:
+ init_tabentry(env);
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ init_funcentry(env);
+ break;
+
+ case PGSTAT_TYPE_DB:
+ init_dbentry(env);
+ dbstat = (PgStat_StatDBEntry *) &env->body;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ break;
+ default:
+ Assert(false);
+ }
+
+ LWLockRelease(&env->lock);
+ }
+
+ pfree(envlist);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1361,29 +1560,37 @@ pgstat_reset_counters(void)
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
+ /* Reset the archiver statistics for the cluster. */
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
+ shared_archiverStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ /* Reset the bgwriter statistics for the cluster. */
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ {
+ TimestampTz now = GetCurrentTimestamp();
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
+ shared_globalStats->stat_reset_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\" or \"bgwriter\".")));
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1392,17 +1599,42 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ env = get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid,
+ false, NULL, NULL);
+ Assert(env);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&env->lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ env = get_stat_entry(stattype, MyDatabaseId, objoid, false, NULL, NULL);
+ LWLockAcquire(&env->lock, LW_EXCLUSIVE);
+ if (env->type == PGSTAT_TYPE_TABLE)
+ init_tabentry(env);
+ else
+ {
+ Assert(env->type == PGSTAT_TYPE_FUNCTION);
+ init_funcentry(env);
+ }
+ LWLockRelease(&env->lock);
}
/* ----------
@@ -1418,15 +1650,27 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_reset_slru_counter(const char *name)
{
- PgStat_MsgResetslrucounter msg;
+ int i;
+ TimestampTz ts = GetCurrentTimestamp();
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSLRUCOUNTER);
- msg.m_index = (name) ? pgstat_slru_index(name) : -1;
-
- pgstat_send(&msg, sizeof(msg));
+ if (name)
+ {
+ i = pgstat_slru_index(name);
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ MemSet(&shared_SLRUStats[i], 0, sizeof(PgStat_StatSLRUEntry));
+ shared_SLRUStats->entry[i].stat_reset_timestamp = ts;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS; i++)
+ {
+ MemSet(&shared_SLRUStats[i], 0, sizeof(PgStat_StatSLRUEntry));
+ shared_SLRUStats->entry[i].stat_reset_timestamp = ts;
+ }
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
@@ -1440,48 +1684,63 @@ pgstat_reset_slru_counter(const char *name)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ ts = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Store the last autovacuum time in the database's hash table entry.
+ */
+ dbentry = get_local_dbstat_entry(dboid);
+ dbentry->last_autovac_time = ts;
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_TableStatus *tabentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+ tabentry = get_local_tabstat_entry(tableoid, shared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->t_counts.autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->t_counts.vacuum_count++;
+ }
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1492,9 +1751,10 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_TableStatus *tabentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1502,10 +1762,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1523,158 +1783,172 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ tabentry = get_local_tabstat_entry(RelationGetRelid(rel),
+ rel->rd_rel->relisshared);
+
+ tabentry->t_counts.t_delta_live_tuples = livetuples;
+ tabentry->t_counts.t_delta_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->t_counts.reset_changed_tuples = true;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->t_counts.analyze_count++;
+ }
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-
-/* --------
- * pgstat_report_checksum_failures_in_db() -
- *
- * Tell the collector about one or more checksum failures.
- * --------
- */
-void
-pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
-{
- PgStat_MsgChecksumFailure msg;
-
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
-
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
-/* ----------
- * pgstat_ping() -
+/* --------
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Send some junk data to the collector to increase traffic.
- * ----------
+ * Reports about one or more checksum failures.
+ * --------
*/
void
-pgstat_ping(void)
+pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgDummy msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ dbentry = get_local_dbstat_entry(dboid);
+
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
{
+ PgStatEnvelope *env;
PgStat_BackendFunctionEntry *htabent;
bool found;
@@ -1685,26 +1959,15 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
+ htabent = (PgStat_BackendFunctionEntry *) &env->body;
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
if (!found)
MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent->f_id = fcinfo->flinfo->fn_oid;
+
fcu->fs = &htabent->f_counts;
/* save stats for this function, later used to compensate for recursion */
@@ -1717,31 +1980,38 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
+ PgStatEnvelope *env;
+
+ env = get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+ if (!env)
return NULL;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ return (PgStat_BackendFunctionEntry *) &env->body;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1782,9 +2052,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1796,8 +2063,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1813,7 +2079,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1829,116 +2096,157 @@ pgstat_initstats(Relation rel)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
}
-/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize body part of the returned envelope.
+ * ----------
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+static PgStatEnvelope *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
+ PgStatHashEntryKey key;
+ PgStatLocalHashEntry *entry;
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ if (pgStatLocalHash == NULL)
{
HASHCTL ctl;
- memset(&ctl, 0, sizeof(ctl));
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = sizeof(PgStatLocalHashEntry);
+
+ pgStatLocalHash = hash_create("Local stat entries",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ entry = hash_search(pgStatLocalHash, &key,
+ create ? HASH_ENTER : HASH_FIND, found);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ {
+ int len = pgstat_localentsize[type];
+
+ entry->env = MemoryContextAlloc(TopMemoryContext,
+ PgStatEnvelopeSize(len));
+ entry->env->type = type;
+ entry->env->len = len;
}
+ return entry->env;
+}
+
+/* ----------
+ * get_local_dbstat_entry() -
+ *
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
+ */
+static PgStat_StatDBEntry *
+get_local_dbstat_entry(Oid dbid)
+{
+ PgStatEnvelope *env;
+ PgStat_StatDBEntry *dbentry;
+ bool found;
+
/*
* Find an entry or create a new one.
*/
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
+ env = get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
if (!found)
{
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
+ dbentry->databaseid = dbid;
+ dbentry->last_autovac_time = 0;
+ dbentry->last_checksum_failure = 0;
+ dbentry->stat_reset_timestamp = 0;
+ dbentry->stats_timestamp = 0;
+ MemSet(&dbentry->counts, 0, sizeof(PgStat_StatDBCounts));
}
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
+ return dbentry;
+}
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
+ */
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
+{
+ PgStatEnvelope *env;
+ PgStat_TableStatus *tabentry;
+ bool found;
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- return entry;
+ tabentry = (PgStat_TableStatus *) &env->body;
+
+ if (!found)
+ {
+ tabentry->t_id = rel_id;
+ tabentry->t_shared = isshared;
+ tabentry->trans = NULL;
+ MemSet(&tabentry->t_counts, 0, sizeof(PgStat_TableCounts));
+ tabentry->vacuum_timestamp = 0;
+ tabentry->autovac_vacuum_timestamp = 0;
+ tabentry->analyze_timestamp = 0;
+ tabentry->autovac_analyze_timestamp = 0;
+ }
+
+ return tabentry;
}
+
/*
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel from the current
+ * database then from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry, return NULL, don't create a new one
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStatEnvelope *env;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!env)
+ env = get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
+ if (env)
+ return (PgStat_TableStatus *) &env->body;
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return NULL;
}
/*
@@ -2369,8 +2677,8 @@ AtPrepare_PgStat(void)
*
* All we need do here is unlink the transaction stats state from the
* nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
+ * reported to the activity stats facility immediately, while the effects on
+ * live and dead tuple counts are preserved in the 2PC state file.
*
* Note: AtEOXact_PgStat is not called during PREPARE.
*/
@@ -2415,7 +2723,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2451,7 +2759,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2468,88 +2776,176 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
}
+/* ----------
+ * snapshot_statentry() -
+ *
+ * Common routine for functions pgstat_fetch_stat_*entry()
+ *
+ * Returns the pointer to the snapshot of the shared entry for the key or NULL
+ * if not found. Returned snapshots are stable during the current transaction
+ * or until pgstat_clear_snapshot() is called.
+ *
+ * Created snapshots are stored in pgStatSnapshotHash.
+ */
+static void *
+snapshot_statentry(const PgStatTypes type, const Oid dbid, const Oid objid)
+{
+ PgStatSnapshot *snap = NULL;
+ bool found;
+ PgStatHashEntryKey key;
+ size_t statentsize = pgstat_entsize[type];
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ /*
+ * Create new hash, with rather arbitrary initial number of entries since
+ * we don't know how this hash will grow.
+ */
+ if (!pgStatSnapshotHash)
+ {
+ HASHCTL ctl;
+
+ /*
+ * Create the hash in the stats context
+ *
+ * The entry is prepended by common header part represented by
+ * PgStatSnapshot.
+ */
+
+ ctl.keysize = sizeof(PgStatHashEntryKey);
+ ctl.entrysize = PgStatSnapshotSize(statentsize);
+ ctl.hcxt = pgStatSnapshotContext;
+ pgStatSnapshotHash = hash_create("pgstat snapshot hash", 32, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ }
+
+ /* Find a snapshot */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ snap = hash_search(pgStatSnapshotHash, &key, HASH_ENTER, &found);
+
+ /*
+ * Refer shared hash if not found in the snapshot hash.
+ *
+ * In transaction state, it is obvious that we should create a snapshot
+ * entriy for consistency. If we are not, we return an up-to-date entry.
+ * Having said that, we need a snapshot since shared stats entry can be
+ * modified anytime. We share the same snapshot entry for the purpose.
+ */
+ if (!found || !IsTransactionState())
+ {
+ PgStatEnvelope *shenv;
+
+ shenv = get_stat_entry(type, dbid, objid, true, NULL, NULL);
+
+ if (shenv)
+ memcpy(&snap->body, &shenv->body, statentsize);
+
+ snap->negative = !shenv;
+ }
+
+ if (snap->negative)
+ return NULL;
+
+ return &snap->body;
+}
+
+
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends. The returned entries are cached
+ * until transaction end or pgstat_clear_snapshot() is called.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ /* If not done for this transaction, take a snapshot of global stats */
+ pgstat_snapshot_global_stats();
+
+ /* caller doesn't have a business with snapshot-local members */
+ return (PgStat_StatDBEntry *)
+ snapshot_statentry(PGSTAT_TYPE_DB, dbid, InvalidOid);
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_snapshot(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_snapshot() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is cached
+ * until transaction end or pgstat_clear_snapshot() is called.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_snapshot(bool shared, Oid reloid)
+{
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ return (PgStat_StatTabEntry *)
+ snapshot_statentry(PGSTAT_TYPE_TABLE, dboid, reloid);
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2563,25 +2959,60 @@ pgstat_fetch_stat_tabentry(Oid relid)
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
- /* load the stats file if needed */
- backend_read_statsfile();
+ return (PgStat_StatFuncEntry *)
+ snapshot_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId, func_id);
+}
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
+/*
+ * pgstat_snapshot_global_stats() -
+ *
+ * Makes a snapshot of global stats if not done yet. They will be kept until
+ * subsequent call of pgstat_clear_snapshot() or the end of the current
+ * memory context (typically TopTransactionContext).
+ * ----------
+ */
+static void
+pgstat_snapshot_global_stats(void)
+{
+ MemoryContext oldcontext;
+ int i;
+
+ attach_shared_stats();
+
+ /* Nothing to do if already done */
+ if (global_snapshot_is_valid)
+ return;
+
+ oldcontext = MemoryContextSwitchTo(pgStatSnapshotContext);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+
+ memcpy(&snapshot_globalStats, shared_globalStats,
+ sizeof(PgStat_GlobalStats));
+ memcpy(&snapshot_archiverStats, shared_archiverStats,
+ sizeof(PgStat_ArchiverStats));
+ memcpy(snapshot_SLRUStats, shared_SLRUStats,
+ sizeof(PgStat_StatSLRUEntry) * SLRU_NUM_ELEMENTS);
+
+ LWLockRelease(StatsLock);
+
+ /* Fill in empty timestamp of SLRU stats */
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS ; i++)
{
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ if (snapshot_SLRUStats[i].stat_reset_timestamp == 0)
+ snapshot_SLRUStats[i].stat_reset_timestamp =
+ snapshot_globalStats.stat_reset_timestamp;
}
+ global_snapshot_is_valid = true;
- return funcentry;
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
}
-
/* ----------
* pgstat_fetch_stat_beentry() -
*
@@ -2652,9 +3083,10 @@ pgstat_fetch_stat_numbackends(void)
PgStat_ArchiverStats *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &archiverStats;
+ return &snapshot_archiverStats;
}
@@ -2669,9 +3101,10 @@ pgstat_fetch_stat_archiver(void)
PgStat_GlobalStats *
pgstat_fetch_global(void)
{
- backend_read_statsfile();
+ /* If not done for this transaction, take a stats snapshot */
+ pgstat_snapshot_global_stats();
- return &globalStats;
+ return &snapshot_globalStats;
}
@@ -2683,12 +3116,12 @@ pgstat_fetch_global(void)
* a pointer to the slru statistics struct.
* ---------
*/
-PgStat_SLRUStats *
+PgStat_StatSLRUEntry *
pgstat_fetch_slru(void)
{
- backend_read_statsfile();
+ pgstat_snapshot_global_stats();
- return slruStats;
+ return snapshot_SLRUStats;
}
@@ -2902,8 +3335,8 @@ pgstat_initialize(void)
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3079,12 +3512,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3097,7 +3533,7 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
@@ -3114,6 +3550,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3374,7 +3812,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3669,9 +4108,6 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
- break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
break;
@@ -4324,94 +4760,71 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
- * ----------
- */
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
-{
- int rc;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- ((PgStat_MsgHdr *) msg)->m_size = len;
-
- /* We'll retry after EINTR, but ignore all other failures */
- do
- {
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
+ * Report archiver statistics
* ----------
*/
void
-pgstat_send_archiver(const char *xlog, bool failed)
+pgstat_report_archiver(const char *xlog, bool failed)
{
- PgStat_MsgArchiver msg;
+ TimestampTz now = GetCurrentTimestamp();
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- strlcpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ if (failed)
+ {
+ /* Failed archival attempt */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->failed_count;
+ memcpy(shared_archiverStats->last_failed_wal, xlog,
+ sizeof(shared_archiverStats->last_failed_wal));
+ shared_archiverStats->last_failed_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
+ else
+ {
+ /* Successful archival operation */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ ++shared_archiverStats->archived_count;
+ memcpy(shared_archiverStats->last_archived_wal, xlog,
+ sizeof(shared_archiverStats->last_archived_wal));
+ shared_archiverStats->last_archived_timestamp = now;
+ LWLockRelease(StatsLock);
+ }
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+
+ PgStat_BgWriter *l = &BgWriterStats;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ shared_globalStats->timed_checkpoints += l->timed_checkpoints;
+ shared_globalStats->requested_checkpoints += l->requested_checkpoints;
+ shared_globalStats->checkpoint_write_time += l->checkpoint_write_time;
+ shared_globalStats->checkpoint_sync_time += l->checkpoint_sync_time;
+ shared_globalStats->buf_written_checkpoints += l->buf_written_checkpoints;
+ shared_globalStats->buf_written_clean += l->buf_written_clean;
+ shared_globalStats->maxwritten_clean += l->maxwritten_clean;
+ shared_globalStats->buf_written_backend += l->buf_written_backend;
+ shared_globalStats->buf_fsync_backend += l->buf_fsync_backend;
+ shared_globalStats->buf_alloc += l->buf_alloc;
+ LWLockRelease(StatsLock);
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4419,477 +4832,31 @@ pgstat_send_bgwriter(void)
MemSet(&BgWriterStats, 0, sizeof(BgWriterStats));
}
-/* ----------
- * pgstat_send_slru() -
- *
- * Send SLRU statistics to the collector
- * ----------
- */
-static void
-pgstat_send_slru(void)
-{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgSLRU all_zeroes;
-
- for (int i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /*
- * This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
- */
- if (memcmp(&SLRUStats[i], &all_zeroes, sizeof(PgStat_MsgSLRU)) == 0)
- continue;
-
- /* set the SLRU type before each send */
- SLRUStats[i].m_index = i;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&SLRUStats[i].m_hdr, PGSTAT_MTYPE_SLRU);
- pgstat_send(&SLRUStats[i], sizeof(PgStat_MsgSLRU));
-
- /*
- * Clear out the statistics buffer, so it can be re-used.
- */
- MemSet(&SLRUStats[i], 0, sizeof(PgStat_MsgSLRU));
- }
-}
-
-
-/* ----------
- * PgstatCollectorMain() -
- *
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
- WaitEvent event;
- WaitEventSet *wes;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /* Prepare to wait for our latch or data in our socket. */
- wes = CreateWaitEventSet(CurrentMemoryContext, 3);
- AddWaitEventToSet(wes, WL_LATCH_SET, PGINVALID_SOCKET, MyLatch, NULL);
- AddWaitEventToSet(wes, WL_POSTMASTER_DEATH, PGINVALID_SOCKET, NULL, NULL);
- AddWaitEventToSet(wes, WL_SOCKET_READABLE, pgStatSock, NULL, NULL);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSLRUCOUNTER:
- pgstat_recv_resetslrucounter(&msg.msg_resetslrucounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_SLRU:
- pgstat_recv_slru(&msg.msg_slru, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitEventSetWait(wes, -1L, &event, 1, WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitEventSetWait(wes, 2 * 1000L /* msec */ , &event, 1,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr == 1 && event.events == WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- FreeWaitEventSet(wes);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
-{
- PgStat_StatDBEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
- */
- if (!found)
- reset_dbentry_counters(result);
-
- return result;
-}
-
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
- */
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
-{
- PgStat_StatTabEntry *result;
- bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
-
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->inserts_since_vacuum = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
-}
-
/* ----------
* pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
- *
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * Write the global statistics file, as well as DB files.
* ----------
*/
-static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+void
+pgstat_write_statsfiles(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
elog(DEBUG2, "writing stats file \"%s\"", statfile);
+ create_missing_dbentries();
+
/*
* Open the statistics temp file to write out the current values.
*/
@@ -4906,7 +4873,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ shared_globalStats->stats_timestamp = GetCurrentTimestamp();
/*
* Write the file header --- currently just a format ID.
@@ -4918,38 +4885,31 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(shared_globalStats, sizeof(*shared_globalStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Write SLRU stats struct
- */
- rc = fwrite(slruStats, sizeof(slruStats), 1, fpout);
+ rc = fwrite(shared_archiverStats, sizeof(*shared_archiverStats), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Walk through the database table.
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_DB, InvalidOid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) &(*penv)->body;
+
/*
* Write out the table and function stats for this DB into the
* appropriate per-DB stat file, if required.
*/
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
- {
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ /* Make DB's timestamp consistent with the global stats */
+ dbentry->stats_timestamp = shared_globalStats->stats_timestamp;
- pgstat_write_db_statsfile(dbentry, permanent);
- }
+ pgstat_write_database_stats(dbentry);
/*
* Write out the DB entry. We don't write the tables or functions
@@ -4960,6 +4920,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
+
/*
* No more output to be done. Close the temp file and replace the old
* pgstat.stat with it. The ferror() check replaces testing for error
@@ -4992,55 +4954,19 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
}
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
+ * pgstat_write_database_stats() -
+ * Write the stat file for a single database.
* ----------
*/
static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
+pgstat_write_database_stats(PgStat_StatDBEntry *dbentry)
{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
+ PgStatEnvelope **envlist;
+ PgStatEnvelope **penv;
FILE *fpout;
int32 format_id;
Oid dbid = dbentry->databaseid;
@@ -5048,8 +4974,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
+ get_dbstat_filename(true, dbid, tmpfile, MAXPGPATH);
+ get_dbstat_filename(false, dbid, statfile, MAXPGPATH);
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -5076,24 +5002,31 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
/*
* Walk through the database's access stats per table.
*/
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_TABLE, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatTabEntry *tabentry = (PgStat_StatTabEntry *) &(*penv)->body;
+
fputc('T', fpout);
rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* Walk through the database's function stats table.
*/
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
+ envlist = collect_stat_entries(PGSTAT_TYPE_FUNCTION, dbentry->databaseid);
+ for (penv = envlist; *penv != NULL; penv++)
{
+ PgStat_StatFuncEntry *funcentry =
+ (PgStat_StatFuncEntry *) &(*penv)->body;
+
fputc('F', fpout);
rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ pfree(envlist);
/*
* No more output to be done. Close the temp file and replace the old
@@ -5127,102 +5060,165 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
+}
- if (permanent)
+/* ----------
+ * create_missing_dbentries() -
+ *
+ * There may be the case where database entry is missing for the database
+ * where object stats are recorded. This function creates such missing
+ * dbentries so that so that all stats entries can be written out to files.
+ * ----------
+ */
+static void
+create_missing_dbentries(void)
+{
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ HTAB *oidhash;
+ HASHCTL ctl;
+ HASH_SEQ_STATUS scan;
+ Oid *poid;
+
+ memset(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(Oid);
+ ctl.hcxt = CurrentMemoryContext;
+ oidhash = hash_create("Temporary table of OIDs",
+ PGSTAT_TABLE_HASH_SIZE,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Collect OID from the shared stats hash */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ hash_search(oidhash, &p->key.databaseid, HASH_ENTER, NULL);
+ dshash_seq_term(&hstat);
+
+ /* Create missing database entries if not exists. */
+ hash_seq_init(&scan, oidhash);
+ while ((poid = (Oid *) hash_seq_search(&scan)) != NULL)
+ (void) get_stat_entry(PGSTAT_TYPE_DB, *poid, InvalidOid,
+ false, init_dbentry, NULL);
+
+ hash_destroy(oidhash);
+}
+
+
+/* ----------
+ * get_stat_entry() -
+ *
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new envelope. If found is not NULL, it is set to true if
+ * existing entry is found or false if not.
+ * ----------
+ */
+static PgStatEnvelope *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool nowait, entry_initializer initfunc, bool *found)
+{
+ bool create = (initfunc != NULL);
+ PgStatHashEntry *shent;
+ PgStatEnvelope *shenv = NULL;
+ PgStatHashEntryKey key;
+ bool myfound;
+
+ Assert(type != PGSTAT_TYPE_ALL);
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ shent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &myfound);
+ if (shent)
{
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
+ if (create && !myfound)
+ {
+ /* Create new stats envelope. */
+ size_t envsize = PgStatEnvelopeSize(pgstat_entsize[type]);
+ dsa_pointer chunk = dsa_allocate0(area, envsize);
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
+ shenv = dsa_get_address(area, chunk);
+ shenv->type = type;
+ shenv->databaseid = dbid;
+ shenv->objectid = objid;
+ shenv->len = pgstat_entsize[type];
+ LWLockInitialize(&shenv->lock, LWTRANCHE_STATS);
+
+ /*
+ * The lock on dshsh is released just after. Call initializer
+ * callback before it is exposed to other process.
+ */
+ if (initfunc)
+ initfunc(shenv);
+
+ /* Link the new entry from the hash entry. */
+ shent->env = chunk;
+ }
+ else
+ shenv = dsa_get_address(area, shent->env);
+
+ dshash_release_lock(pgStatSharedHash, shent);
}
+
+ if (found)
+ *found = myfound;
+
+ return shenv;
}
+
/* ----------
* pgstat_read_statsfiles() -
*
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * Reads in existing activity statistics files into the shared stats hash.
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
- *
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+void
+pgstat_read_statsfiles(void)
{
+ PgStatEnvelope *env;
PgStat_StatDBEntry *dbentry;
PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
- int i;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * Clear out global and archiver statistics so they start from zero in
- * case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
- memset(&slruStats, 0, sizeof(slruStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all SLRU items too.
- */
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- slruStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ shared_globalStats->stat_reset_timestamp = GetCurrentTimestamp();
+ shared_archiverStats->stat_reset_timestamp =
+ shared_globalStats->stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5231,7 +5227,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
@@ -5239,49 +5235,30 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (fread(shared_globalStats, 1, sizeof(*shared_globalStats), fpin) !=
+ sizeof(*shared_globalStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
+ MemSet(shared_globalStats, 0, sizeof(*shared_globalStats));
goto done;
}
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (fread(shared_archiverStats, 1, sizeof(*shared_archiverStats), fpin) !=
+ sizeof(*shared_archiverStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
+ MemSet(shared_archiverStats, 0, sizeof(*shared_archiverStats));
goto done;
}
/*
- * Read SLRU stats struct
- */
- if (fread(slruStats, 1, sizeof(slruStats), fpin) != sizeof(slruStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&slruStats, 0, sizeof(slruStats));
- goto done;
- }
-
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5295,7 +5272,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
fpin) != offsetof(PgStat_StatDBEntry, tables))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5304,76 +5281,33 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Add to the DB hash
*/
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
+
+ env = get_stat_entry(PGSTAT_TYPE_DB, dbbuf.databaseid,
+ InvalidOid,
+ false, init_dbentry, &found);
+ dbentry = (PgStat_StatDBEntry *) &env->body;
+
+ /* don't allow duplicate dbentries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- memset(&hash_ctl, 0, sizeof(hash_ctl));
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
+ memcpy(dbentry, &dbbuf,
+ offsetof(PgStat_StatDBEntry, tables));
+ /* Read the data from the database-specific file. */
+ pgstat_read_db_statsfile(dbentry);
break;
case 'E':
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5383,59 +5317,50 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
- return dbhash;
+ return;
}
/* ----------
* pgstat_read_db_statsfile() -
*
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
+ * Reads in the at-rest statistics file and create shared statistics
+ * tables. The file is removed after reading.
* ----------
*/
static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
+pgstat_read_db_statsfile(PgStat_StatDBEntry *dbentry)
{
+ PgStatEnvelope *env;
PgStat_StatTabEntry *tabentry;
PgStat_StatTabEntry tabbuf;
PgStat_StatFuncEntry funcbuf;
PgStat_StatFuncEntry *funcentry;
+ dshash_table *tabhash = NULL;
+ dshash_table *funchash = NULL;
FILE *fpin;
int32 format_id;
bool found;
char statfile[MAXPGPATH];
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
+ get_dbstat_filename(false, dbentry->databaseid, statfile, MAXPGPATH);
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
@@ -5448,14 +5373,14 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
for (;;)
{
@@ -5468,25 +5393,21 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
fpin) != sizeof(PgStat_StatTabEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE,
+ dbentry->databaseid, tabbuf.tableid,
+ false, init_tabentry, &found);
+ tabentry = (PgStat_StatTabEntry *) &env->body;
+ /* don't allow duplicate entries */
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5502,25 +5423,20 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
fpin) != sizeof(PgStat_StatFuncEntry))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
}
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
+ env = get_stat_entry(PGSTAT_TYPE_TABLE, dbentry->databaseid,
+ funcbuf.functionid,
+ false, init_funcentry, &found);
+ funcentry = (PgStat_StatFuncEntry *) &env->body;
if (found)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5536,7 +5452,7 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
goto done;
default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"",
statfile)));
goto done;
@@ -5544,304 +5460,15 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read archiver stats struct
- */
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read SLRU stats struct
- */
- if (fread(mySLRUStats, 1, sizeof(mySLRUStats), fpin) != sizeof(mySLRUStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
+ if (tabhash)
+ dshash_detach(tabhash);
+ if (funchash)
+ dshash_detach(funchash);
-done:
FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
- /*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
- */
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
- {
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
-
- CHECK_FOR_INTERRUPTS();
-
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
- {
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
- }
-
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG, "stats collector's time %s is later than backend local time %s",
- filetime, mytime);
- pfree(filetime);
- pfree(mytime);
- }
-
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
- }
-
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
-
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
- }
-
- if (count >= PGSTAT_POLL_LOOP_COUNT)
- ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
-
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
-}
-
-/* ----------
- * pgstat_setup_memcxt() -
- *
- * Create pgStatLocalContext, if not already done.
- * ----------
- */
-static void
-pgstat_setup_memcxt(void)
-{
- if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
}
@@ -5860,797 +5487,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- elog(LOG,
- "stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid);
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum = tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- tabentry->inserts_since_vacuum = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum += tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
}
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
+ if (pgStatSnapshotContext)
{
- char statfile[MAXPGPATH];
+ MemoryContextReset(pgStatSnapshotContext);
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
+ /* Reset variables that pointed to the context */
+ global_snapshot_is_valid = false;
+ pgStatSnapshotHash = NULL;
}
}
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_resetslrucounter() -
- *
- * Reset some SLRU statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len)
-{
- int i;
- TimestampTz ts = GetCurrentTimestamp();
-
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /* reset entry with the given index, or all entries (index is -1) */
- if ((msg->m_index == -1) || (msg->m_index == i))
- {
- memset(&slruStats[i], 0, sizeof(slruStats[i]));
- slruStats[i].stat_reset_timestamp = ts;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signaling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * It is quite possible that a non-aggressive VACUUM ended up skipping
- * various pages, however, we'll zero the insert counter here regardless.
- * It's currently used only to track when we need to perform an "insert"
- * autovacuum, which are mainly intended to freeze newly inserted tuples.
- * Zeroing this may just mean we'll not try to vacuum the table again
- * until enough tuples have been inserted to trigger another insert
- * autovacuum. An anti-wraparound autovacuum will catch any persistent
- * stragglers.
- */
- tabentry->inserts_since_vacuum = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_slru() -
- *
- * Process a SLRU message.
- * ----------
- */
-static void
-pgstat_recv_slru(PgStat_MsgSLRU *msg, int len)
-{
- slruStats[msg->m_index].blocks_zeroed += msg->m_blocks_zeroed;
- slruStats[msg->m_index].blocks_hit += msg->m_blocks_hit;
- slruStats[msg->m_index].blocks_read += msg->m_blocks_read;
- slruStats[msg->m_index].blocks_written += msg->m_blocks_written;
- slruStats[msg->m_index].blocks_exists += msg->m_blocks_exists;
- slruStats[msg->m_index].flush += msg->m_flush;
- slruStats[msg->m_index].truncate += msg->m_truncate;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
-}
-
/*
* Convert a potentially unsafely truncated activity string (see
* PgBackendStatus.st_activity_raw's documentation) into a correctly truncated
@@ -6739,7 +5594,7 @@ pgstat_slru_name(int slru_idx)
* Returns pointer to entry with counters for given SLRU (based on the name
* stored in SlruCtl as lwlock tranche name).
*/
-static inline PgStat_MsgSLRU *
+static PgStat_StatSLRUEntry *
slru_entry(int slru_idx)
{
/*
@@ -6750,7 +5605,7 @@ slru_entry(int slru_idx)
Assert((slru_idx >= 0) && (slru_idx < SLRU_NUM_ELEMENTS));
- return &SLRUStats[slru_idx];
+ return &local_SLRUStats[slru_idx];
}
/*
@@ -6760,41 +5615,48 @@ slru_entry(int slru_idx)
void
pgstat_count_slru_page_zeroed(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_zeroed += 1;
+ slru_entry(slru_idx)->blocks_zeroed += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_hit(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_hit += 1;
+ slru_entry(slru_idx)->blocks_hit += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_exists(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_exists += 1;
+ slru_entry(slru_idx)->blocks_exists += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_read(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_read += 1;
+ slru_entry(slru_idx)->blocks_read += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_written(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_written += 1;
+ slru_entry(slru_idx)->blocks_written += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_flush(int slru_idx)
{
- slru_entry(slru_idx)->m_flush += 1;
+ slru_entry(slru_idx)->flush += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_truncate(int slru_idx)
{
- slru_entry(slru_idx)->m_truncate += 1;
+ slru_entry(slru_idx)->truncate += 1;
+ have_slrustats = true;
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4c296dd214..d10bba75f0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -256,7 +256,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -517,7 +516,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1339,12 +1337,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1793,11 +1785,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2725,8 +2712,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3053,8 +3038,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3121,13 +3104,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3200,22 +3176,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3676,22 +3636,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3911,8 +3855,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3936,8 +3878,7 @@ PostmasterStateMachine(void)
{
/*
* PM_WAIT_DEAD_END state ends when the BackendList is entirely empty
- * (ie, no dead_end children remain), and the archiver and stats
- * collector are gone too.
+ * (ie, no dead_end children remain), and the archiveris gone too.
*
* The reason we wait for those two is to protect them against a new
* postmaster starting conflicting subprocesses; this isn't an
@@ -3947,8 +3888,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4149,8 +4089,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5123,18 +5061,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5253,12 +5179,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6159,7 +6079,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6215,8 +6134,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6451,7 +6368,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 6064384e32..cc6b2bb5de 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1553,8 +1553,8 @@ is_checksummed_file(const char *fullpath, const char *filename)
*
* If 'missing_ok' is true, will not throw an error if the file is not found.
*
- * If dboid is anything other than InvalidOid then any checksum failures detected
- * will get reported to the stats collector.
+ * If dboid is anything other than InvalidOid then any checksum failures
+ * detected will get reported to the activity stats facility.
*
* Returns true if the file was successfully sent, false if 'missing_ok',
* and the file did not exist.
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index a2a963bd5b..61fa52ed66 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2045,7 +2045,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ BgWriterStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2155,7 +2155,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2345,7 +2345,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2353,7 +2353,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 96c2aaabbd..55693cfa3a 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -149,6 +149,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -267,6 +268,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 2fa90cc095..17a46db74b 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -177,7 +177,9 @@ static const char *const BuiltinTrancheNames[] = {
/* LWTRANCHE_PARALLEL_APPEND: */
"ParallelAppend",
/* LWTRANCHE_PER_XACT_PREDICATE_LIST: */
- "PerXactPredicateList"
+ "PerXactPredicateList",
+ /* LWTRANCHE_STATS: */
+ "ActivityStatistics"
};
StaticAssertDecl(lengthof(BuiltinTrancheNames) ==
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index 774292fd94..91bf8d5b5d 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -53,3 +53,4 @@ XactTruncationLock 44
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+StatsLock 48
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index dcc09df0c7..0dec4b9145 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -415,8 +415,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
DropRelFileNodesAllBuffers(rnodes, nrels);
/*
- * It'd be nice to tell the stats collector to forget them immediately,
- * too. But we can't because we don't know the OIDs.
+ * It'd be nice to tell the activity stats facility to forget them
+ * immediately, too. But we can't because we don't know the OIDs.
*/
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index c9424f167c..0396aa469f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3209,6 +3209,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3783,6 +3789,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_stats_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4185,11 +4192,12 @@ PostgresMain(int argc, char *argv[],
* Note: this includes fflush()'ing the last of the prior output.
*
* This is also a good time to send collected statistics to the
- * collector, and to update the PS stats display. We avoid doing
- * those every time through the message loop because it'd slow down
- * processing of batched messages, and because we don't want to report
- * uncommitted updates (that confuses autovacuum). The notification
- * processor wants a call too, if we are not in a transaction block.
+ * activity statistics, and to update the PS stats display. We avoid
+ * doing those every time through the message loop because it'd slow
+ * down processing of batched messages, and because we don't want to
+ * report uncommitted updates (that confuses autovacuum). The
+ * notification processor wants a call too, if we are not in a
+ * transaction block.
*/
if (send_ready_for_query)
{
@@ -4221,6 +4229,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4233,8 +4243,13 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ disable_idle_stats_update_timeout = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4269,7 +4284,7 @@ PostgresMain(int argc, char *argv[],
DoingCommandRead = false;
/*
- * (5) turn off the idle-in-transaction timeout
+ * (5) turn off the idle-in-transaction timeout and stats update timeout
*/
if (disable_idle_in_transaction_timeout)
{
@@ -4277,6 +4292,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_stats_update_timeout)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ disable_idle_stats_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 95738a4e34..3e8ce0b3bf 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1,7 +1,7 @@
/*-------------------------------------------------------------------------
*
* pgstatfuncs.c
- * Functions for accessing the statistics collector data
+ * Functions for accessing the activity statistics data
*
* Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -35,9 +35,6 @@
#define HAS_PGSTAT_PERMISSIONS(role) (is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS) || has_privs_of_role(GetUserId(), role))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1267,7 +1264,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1283,7 +1280,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1299,7 +1296,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1315,7 +1312,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1331,7 +1328,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1347,7 +1344,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1363,7 +1360,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1379,7 +1376,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1395,7 +1392,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1428,7 +1425,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1444,7 +1441,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1459,7 +1456,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1474,7 +1471,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1489,7 +1486,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1504,7 +1501,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1519,7 +1516,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1534,11 +1531,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1553,7 +1550,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1571,7 +1568,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1608,7 +1605,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1624,7 +1621,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1710,7 +1707,7 @@ pg_stat_get_slru(PG_FUNCTION_ARGS)
MemoryContext per_query_ctx;
MemoryContext oldcontext;
int i;
- PgStat_SLRUStats *stats;
+ PgStat_StatSLRUEntry *stats;
/* check to see if caller supports us returning a tuplestore */
if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
@@ -1744,7 +1741,7 @@ pg_stat_get_slru(PG_FUNCTION_ARGS)
/* for each row */
Datum values[PG_STAT_GET_SLRU_COLS];
bool nulls[PG_STAT_GET_SLRU_COLS];
- PgStat_SLRUStats stat = stats[i];
+ PgStat_StatSLRUEntry stat = stats[i];
const char *name;
name = pgstat_slru_name(i);
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 6ab8216839..883c7f8802 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -33,6 +33,7 @@ volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index cf8f9579c3..ecd0cec2c3 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -243,9 +243,6 @@ GetBackendTypeDesc(BackendType backendType)
case B_ARCHIVER:
backendDesc = "archiver";
break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
case B_LOGGER:
backendDesc = "logger";
break;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index d4ab4c7e23..4ff4cc33d9 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -631,6 +632,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1245,6 +1248,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index de87ad6ef7..36113378f7 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -743,8 +743,8 @@ const char *const config_group_names[] =
gettext_noop("Statistics"),
/* STATS_MONITORING */
gettext_noop("Statistics / Monitoring"),
- /* STATS_COLLECTOR */
- gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* STATS_ACTIVITY */
+ gettext_noop("Statistics / Query and Index Activity Statistics"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
@@ -1454,7 +1454,7 @@ static struct config_bool ConfigureNamesBool[] =
#endif
{
- {"track_activities", PGC_SUSET, STATS_COLLECTOR,
+ {"track_activities", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects information about executing commands."),
gettext_noop("Enables the collection of information on the currently "
"executing command of each session, along with "
@@ -1465,7 +1465,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_counts", PGC_SUSET, STATS_COLLECTOR,
+ {"track_counts", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects statistics on database activity."),
NULL
},
@@ -1474,7 +1474,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_io_timing", PGC_SUSET, STATS_COLLECTOR,
+ {"track_io_timing", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects timing statistics for database I/O activity."),
NULL
},
@@ -4310,7 +4310,7 @@ static struct config_string ConfigureNamesString[] =
},
{
- {"stats_temp_directory", PGC_SIGHUP, STATS_COLLECTOR,
+ {"stats_temp_directory", PGC_SIGHUP, STATS_ACTIVITY,
gettext_noop("Writes temporary statistics files to the specified directory."),
NULL,
GUC_SUPERUSER_ONLY
@@ -4646,7 +4646,7 @@ static struct config_enum ConfigureNamesEnum[] =
},
{
- {"track_functions", PGC_SUSET, STATS_COLLECTOR,
+ {"track_functions", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects function-level statistics on database activity."),
NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 9cb571f7cc..668a2d033a 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -579,7 +579,7 @@
# STATISTICS
#------------------------------------------------------------------------------
-# - Query and Index Statistics Collector -
+# - Query and Index Activity Statistics -
#track_activities = on
#track_counts = on
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index f674a7c94e..26603e95e4 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -124,7 +124,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index bba002ad24..b70c495f2a 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -83,6 +83,8 @@ extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -321,7 +323,6 @@ typedef enum BackendType
B_WAL_SENDER,
B_WAL_WRITER,
B_ARCHIVER,
- B_STATS_COLLECTOR,
B_LOGGER,
} BackendType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 0dfbac46b4..29a8737498 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2020, PostgreSQL Global Development Group
*
@@ -15,9 +15,11 @@
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -41,35 +43,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_RESETSLRUCOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_SLRU,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -80,9 +53,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -113,18 +85,17 @@ typedef struct PgStat_TableCounts
PgStat_Counter t_delta_live_tuples;
PgStat_Counter t_delta_dead_tuples;
PgStat_Counter t_changed_tuples;
+ bool reset_changed_tuples;
PgStat_Counter t_blocks_fetched;
PgStat_Counter t_blocks_hit;
+
+ PgStat_Counter vacuum_count;
+ PgStat_Counter autovac_vacuum_count;
+ PgStat_Counter analyze_count;
+ PgStat_Counter autovac_analyze_count;
} PgStat_TableCounts;
-/* Possible targets for resetting cluster-wide shared values */
-typedef enum PgStat_Shared_Reset_Target
-{
- RESET_ARCHIVER,
- RESET_BGWRITER
-} PgStat_Shared_Reset_Target;
-
/* Possible object types for resetting single counters */
typedef enum PgStat_Single_Reset_Type
{
@@ -158,6 +129,10 @@ typedef struct PgStat_TableStatus
Oid t_id; /* table's OID */
bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
} PgStat_TableStatus;
@@ -183,308 +158,32 @@ typedef struct PgStat_TableXactStatus
} PgStat_TableXactStatus;
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
- */
-typedef struct PgStat_MsgHdr
-{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
-
-/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
- * ----------
- */
-typedef struct PgStat_MsgDummy
-{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
-
-/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
- * ----------
- */
-
-typedef struct PgStat_MsgInquiry
-{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgResetslrucounter Sent by the backend to tell the collector
- * to reset a SLRU counter
- * ----------
- */
-typedef struct PgStat_MsgResetslrucounter
-{
- PgStat_MsgHdr m_hdr;
- int m_index;
-} PgStat_MsgResetslrucounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgSLRU Sent by a backend to update SLRU statistics.
- * ----------
- */
-typedef struct PgStat_MsgSLRU
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_index;
- PgStat_Counter m_blocks_zeroed;
- PgStat_Counter m_blocks_hit;
- PgStat_Counter m_blocks_read;
- PgStat_Counter m_blocks_written;
- PgStat_Counter m_blocks_exists;
- PgStat_Counter m_flush;
- PgStat_Counter m_truncate;
-} PgStat_MsgSLRU;
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgTempFile
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter buf_alloc;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_BgWriter;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -516,98 +215,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgResetslrucounter msg_resetslrucounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgSLRU msg_slru;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -616,13 +225,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9D
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -632,7 +237,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -642,29 +246,72 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ Oid databaseid;
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/*
+ * SLRU statistics kept in the stats collector
+ */
+typedef struct PgStat_StatSLRUEntry
+{
+ PgStat_Counter blocks_zeroed;
+ PgStat_Counter blocks_hit;
+ PgStat_Counter blocks_read;
+ PgStat_Counter blocks_written;
+ PgStat_Counter blocks_exists;
+ PgStat_Counter flush;
+ PgStat_Counter truncate;
+ TimestampTz stat_reset_timestamp;
+} PgStat_StatSLRUEntry;
+
+
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_HashMountInfo dshash mount information
+ * ----------
+ */
+typedef struct PgStat_HashMountInfo
+{
+ HTAB *snapshot_tables; /* table entry snapshot */
+ HTAB *snapshot_functions; /* function entry snapshot */
+ dshash_table *dshash_tables; /* attached tables dshash */
+ dshash_table *dshash_functions; /* attached functions dshash */
+} PgStat_HashMountInfo;
+
+/* ----------
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
Oid tableid;
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -684,19 +331,15 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
@@ -709,9 +352,8 @@ typedef struct PgStat_StatFuncEntry
PgStat_Counter f_self_time;
} PgStat_StatFuncEntry;
-
/*
- * Archiver statistics kept in the stats collector
+ * Archiver statistics kept in the shared stats
*/
typedef struct PgStat_ArchiverStats
{
@@ -727,7 +369,7 @@ typedef struct PgStat_ArchiverStats
} PgStat_ArchiverStats;
/*
- * Global statistics kept in the stats collector
+ * Global statistics kept in the shared stats
*/
typedef struct PgStat_GlobalStats
{
@@ -745,21 +387,6 @@ typedef struct PgStat_GlobalStats
TimestampTz stat_reset_timestamp;
} PgStat_GlobalStats;
-/*
- * SLRU statistics kept in the stats collector
- */
-typedef struct PgStat_SLRUStats
-{
- PgStat_Counter blocks_zeroed;
- PgStat_Counter blocks_hit;
- PgStat_Counter blocks_read;
- PgStat_Counter blocks_written;
- PgStat_Counter blocks_exists;
- PgStat_Counter flush;
- PgStat_Counter truncate;
- TimestampTz stat_reset_timestamp;
-} PgStat_SLRUStats;
-
/* ----------
* Backend states
@@ -808,7 +435,6 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1060,7 +686,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1257,13 +883,15 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1278,29 +906,26 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* File input/output functions */
+extern void pgstat_read_statsfiles(void);
+extern void pgstat_write_statsfiles(void);
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
@@ -1462,8 +1087,8 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1472,13 +1097,16 @@ extern void pgstat_send_bgwriter(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_snapshot(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
-extern PgStat_SLRUStats *pgstat_fetch_slru(void);
+extern PgStat_StatSLRUEntry *pgstat_fetch_slru(void);
extern void pgstat_count_slru_page_zeroed(int slru_idx);
extern void pgstat_count_slru_page_hit(int slru_idx);
@@ -1489,5 +1117,6 @@ extern void pgstat_count_slru_flush(int slru_idx);
extern void pgstat_count_slru_truncate(int slru_idx);
extern const char *pgstat_slru_name(int slru_idx);
extern int pgstat_slru_index(const char *name);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index af9b41795d..621e074111 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -218,6 +218,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_SHARED_TIDBITMAP,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_PER_XACT_PREDICATE_LIST,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index 04431d0eb2..3b03464a1a 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -88,7 +88,7 @@ enum config_group
PROCESS_TITLE,
STATS,
STATS_MONITORING,
- STATS_COLLECTOR,
+ STATS_ACTIVITY,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 83a15f6795..77d1572a99 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.18.4
v36-0006-Doc-part-of-shared-memory-based-stats-collector.patch (text/x-patch, 20.7K) download
From 9016c2aaadb8c82d3292ad729b8e75c8c5a4d340 Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Thu, 19 Mar 2020 15:11:09 +0900
Subject: [PATCH v36 6/7] Doc part of shared-memory based stats collector.
---
doc/src/sgml/catalogs.sgml | 6 +-
doc/src/sgml/config.sgml | 34 ++++----
doc/src/sgml/high-availability.sgml | 13 +--
doc/src/sgml/monitoring.sgml | 127 +++++++++++++---------------
doc/src/sgml/ref/pg_dump.sgml | 9 +-
5 files changed, 90 insertions(+), 99 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 508bea3bc6..d30491d4f6 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9208,9 +9208,9 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<para>
<xref linkend="view-table"/> lists the system views described here.
More detailed documentation of each view follows below.
- There are some additional views that provide access to the results of
- the statistics collector; they are described in <xref
- linkend="monitoring-stats-views-table"/>.
+ There are some additional views that provide access to the activity
+ statistics; they are described in
+ <xref linkend="monitoring-stats-views-table"/>.
</para>
<para>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index c4ba49ffaf..530f41c194 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -7190,11 +7190,11 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
<title>Run-time Statistics</title>
<sect2 id="runtime-config-statistics-collector">
- <title>Query and Index Statistics Collector</title>
+ <title>Query and Index Activity Statistics</title>
<para>
- These parameters control server-wide statistics collection features.
- When statistics collection is enabled, the data that is produced can be
+ These parameters control server-wide activity statistics features.
+ When activity statistics is enabled, the data that is produced can be
accessed via the <structname>pg_stat</structname> and
<structname>pg_statio</structname> family of system views.
Refer to <xref linkend="monitoring"/> for more information.
@@ -7210,14 +7210,13 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</term>
<listitem>
<para>
- Enables the collection of information on the currently
- executing command of each session, along with the time when
- that command began execution. This parameter is on by
- default. Note that even when enabled, this information is not
- visible to all users, only to superusers and the user owning
- the session being reported on, so it should not represent a
- security risk.
- Only superusers can change this setting.
+ Enables activity tracking on the currently executing command of
+ each session, along with the time when that command began
+ execution. This parameter is on by default. Note that even when
+ enabled, this information is not visible to all users, only to
+ superusers and the user owning the session being reported on, so it
+ should not represent a security risk. Only superusers can change this
+ setting.
</para>
</listitem>
</varlistentry>
@@ -7248,9 +7247,9 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</term>
<listitem>
<para>
- Enables collection of statistics on database activity.
+ Enables tracking of database activity.
This parameter is on by default, because the autovacuum
- daemon needs the collected information.
+ daemon needs the activity information.
Only superusers can change this setting.
</para>
</listitem>
@@ -8311,7 +8310,7 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
<listitem>
<para>
Specifies the fraction of the total number of heap tuples counted in
- the previous statistics collection that can be inserted without
+ the previously collected statistics that can be inserted without
incurring an index scan at the <command>VACUUM</command> cleanup stage.
This setting currently applies to B-tree indexes only.
</para>
@@ -8323,9 +8322,10 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
the index contains deleted pages that can be recycled during cleanup.
Index statistics are considered to be stale if the number of newly
inserted tuples exceeds the <varname>vacuum_cleanup_index_scale_factor</varname>
- fraction of the total number of heap tuples detected by the previous
- statistics collection. The total number of heap tuples is stored in
- the index meta-page. Note that the meta-page does not include this data
+
+ fraction of the total number of heap tuples in the previously
+ collected statistics. The total number of heap tuples is stored in the
+ index meta-page. Note that the meta-page does not include this data
until <command>VACUUM</command> finds no dead tuples, so B-tree index
scan at the cleanup stage can only be skipped if the second and
subsequent <command>VACUUM</command> cycles detect no dead tuples.
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index beb309e668..45ec7ce68f 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -2366,12 +2366,13 @@ LOG: database system is ready to accept read only connections
</para>
<para>
- The statistics collector is active during recovery. All scans, reads, blocks,
- index usage, etc., will be recorded normally on the standby. Replayed
- actions will not duplicate their effects on primary, so replaying an
- insert will not increment the Inserts column of pg_stat_user_tables.
- The stats file is deleted at the start of recovery, so stats from primary
- and standby will differ; this is considered a feature, not a bug.
+ The activity statistics is collected during recovery. All scans, reads,
+ blocks, index usage, etc., will be recorded normally on the
+ standby. Replayed actions will not duplicate their effects on primary, so
+ replaying an insert will not increment the Inserts column of
+ pg_stat_user_tables. The activity statistics is reset at the start of
+ recovery, so stats from primary and standby will differ; this is
+ considered a feature, not a bug.
</para>
<para>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 673a0e73e4..dbf439891d 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -22,7 +22,7 @@
<para>
Several tools are available for monitoring database activity and
analyzing performance. Most of this chapter is devoted to describing
- <productname>PostgreSQL</productname>'s statistics collector,
+ <productname>PostgreSQL</productname>'s activity statistics,
but one should not neglect regular Unix monitoring programs such as
<command>ps</command>, <command>top</command>, <command>iostat</command>, and <command>vmstat</command>.
Also, once one has identified a
@@ -53,7 +53,6 @@ postgres 15554 0.0 0.0 57536 1184 ? Ss 18:02 0:00 postgres: back
postgres 15555 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: checkpointer
postgres 15556 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: walwriter
postgres 15557 0.0 0.0 58504 2244 ? Ss 18:02 0:00 postgres: autovacuum launcher
-postgres 15558 0.0 0.0 17512 1068 ? Ss 18:02 0:00 postgres: stats collector
postgres 15582 0.0 0.0 58772 3080 ? Ss 18:04 0:00 postgres: joe runbug 127.0.0.1 idle
postgres 15606 0.0 0.0 58772 3052 ? Ss 18:07 0:00 postgres: tgl regression [local] SELECT waiting
postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl regression [local] idle in transaction
@@ -65,9 +64,8 @@ postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl
primary server process. The command arguments
shown for it are the same ones used when it was launched. The next five
processes are background worker processes automatically launched by the
- primary process. (The <quote>stats collector</quote> process will not be present
- if you have set the system not to start the statistics collector; likewise
- the <quote>autovacuum launcher</quote> process can be disabled.)
+ primary process. (The <quote>autovacuum launcher</quote> process will not
+ be present if you have set the system not to start it.)
Each of the remaining
processes is a server process handling one client connection. Each such
process sets its command line display in the form
@@ -130,20 +128,21 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</sect1>
<sect1 id="monitoring-stats">
- <title>The Statistics Collector</title>
+ <title>The Activity Statistics</title>
<indexterm zone="monitoring-stats">
<primary>statistics</primary>
</indexterm>
<para>
- <productname>PostgreSQL</productname>'s <firstterm>statistics collector</firstterm>
- is a subsystem that supports collection and reporting of information about
- server activity. Presently, the collector can count accesses to tables
- and indexes in both disk-block and individual-row terms. It also tracks
- the total number of rows in each table, and information about vacuum and
- analyze actions for each table. It can also count calls to user-defined
- functions and the total time spent in each one.
+ <productname>PostgreSQL</productname>'s <firstterm>activity
+ statistics</firstterm> is a subsystem that supports tracking and reporting
+ of information about server activity. Presently, the activity statistics
+ tracks the count of accesses to tables and indexes in both disk-block and
+ individual-row terms. It also tracks the total number of rows in each
+ table, and information about vacuum and analyze actions for each table. It
+ can also track calls to user-defined functions and the total time spent in
+ each one.
</para>
<para>
@@ -151,15 +150,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
information about exactly what is going on in the system right now, such as
the exact command currently being executed by other server processes, and
which other connections exist in the system. This facility is independent
- of the collector process.
+ of the activity statistics.
</para>
<sect2 id="monitoring-stats-setup">
- <title>Statistics Collection Configuration</title>
+ <title>Activity Statistics Configuration</title>
<para>
- Since collection of statistics adds some overhead to query execution,
- the system can be configured to collect or not collect information.
+ Since tracking for the activity statistics adds some overhead to query
+ execution, the system can be configured to track or not track activity.
This is controlled by configuration parameters that are normally set in
<filename>postgresql.conf</filename>. (See <xref linkend="runtime-config"/> for
details about setting configuration parameters.)
@@ -172,7 +171,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<para>
The parameter <xref linkend="guc-track-counts"/> controls whether
- statistics are collected about table and index accesses.
+ to track activity about table and index accesses.
</para>
<para>
@@ -196,18 +195,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para>
<para>
- The statistics collector transmits the collected information to other
- <productname>PostgreSQL</productname> processes through temporary files.
- These files are stored in the directory named by the
- <xref linkend="guc-stats-temp-directory"/> parameter,
- <filename>pg_stat_tmp</filename> by default.
- For better performance, <varname>stats_temp_directory</varname> can be
- pointed at a RAM-based file system, decreasing physical I/O requirements.
- When the server shuts down cleanly, a permanent copy of the statistics
- data is stored in the <filename>pg_stat</filename> subdirectory, so that
- statistics can be retained across server restarts. When recovery is
- performed at server start (e.g., after immediate shutdown, server crash,
- and point-in-time recovery), all statistics counters are reset.
+ down cleanly, a permanent copy of the statistics data is stored in
+ the <filename>pg_stat</filename> subdirectory, so that statistics can be
+ retained across server restarts. When recovery is performed at server
+ start (e.g. after immediate shutdown, server crash, and point-in-time
+ recovery), all statistics counters are reset.
</para>
</sect2>
@@ -220,48 +212,46 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
linkend="monitoring-stats-dynamic-views-table"/>, are available to show
the current state of the system. There are also several other
views, listed in <xref
- linkend="monitoring-stats-views-table"/>, available to show the results
- of statistics collection. Alternatively, one can
- build custom views using the underlying statistics functions, as discussed
- in <xref linkend="monitoring-stats-functions"/>.
+ linkend="monitoring-stats-views-table"/>, available to show the activity
+ statistics. Alternatively, one can build custom views using the underlying
+ statistics functions, as discussed in
+ <xref linkend="monitoring-stats-functions"/>.
</para>
<para>
- When using the statistics to monitor collected data, it is important
- to realize that the information does not update instantaneously.
- Each individual server process transmits new statistical counts to
- the collector just before going idle; so a query or transaction still in
- progress does not affect the displayed totals. Also, the collector itself
- emits a new report at most once per <varname>PGSTAT_STAT_INTERVAL</varname>
- milliseconds (500 ms unless altered while building the server). So the
- displayed information lags behind actual activity. However, current-query
- information collected by <varname>track_activities</varname> is
- always up-to-date.
+ When using the activity statistics, it is important to realize that the
+ information does not update instantaneously. Each individual server writes
+ out new statistical counts just before going idle, not frequent than once
+ per <varname>PGSTAT_STAT_INTERVAL</varname> milliseconds (1 second unless
+ altered while building the server); so a query or transaction still in
+ progress does not affect the displayed totals. However, current-query
+ information tracked by <varname>track_activities</varname> is always
+ up-to-date.
</para>
<para>
Another important point is that when a server process is asked to display
- any of these statistics, it first fetches the most recent report emitted by
- the collector process and then continues to use this snapshot for all
- statistical views and functions until the end of its current transaction.
- So the statistics will show static information as long as you continue the
- current transaction. Similarly, information about the current queries of
- all sessions is collected when any such information is first requested
- within a transaction, and the same information will be displayed throughout
- the transaction.
- This is a feature, not a bug, because it allows you to perform several
- queries on the statistics and correlate the results without worrying that
- the numbers are changing underneath you. But if you want to see new
- results with each query, be sure to do the queries outside any transaction
- block. Alternatively, you can invoke
+ any of these statistics, it first reads the current statistics and then
+ continues to use this snapshot for all statistical views and functions
+ until the end of its current transaction. So the statistics will show
+ static information as long as you continue the current transaction.
+ Similarly, information about the current queries of all sessions is tracked
+ when any such information is first requested within a transaction, and the
+ same information will be displayed throughout the transaction. This is a
+ feature, not a bug, because it allows you to perform several queries on the
+ statistics and correlate the results without worrying that the numbers are
+ changing underneath you. But if you want to see new results with each
+ query, be sure to do the queries outside any transaction block.
+ Alternatively, you can invoke
<function>pg_stat_clear_snapshot</function>(), which will discard the
current transaction's statistics snapshot (if any). The next use of
statistical information will cause a new snapshot to be fetched.
</para>
-
+
<para>
- A transaction can also see its own statistics (as yet untransmitted to the
- collector) in the views <structname>pg_stat_xact_all_tables</structname>,
+ A transaction can also see its own statistics (as yet unwritten to the
+ server-wide activity statistics) in the
+ views <structname>pg_stat_xact_all_tables</structname>,
<structname>pg_stat_xact_sys_tables</structname>,
<structname>pg_stat_xact_user_tables</structname>, and
<structname>pg_stat_xact_user_functions</structname>. These numbers do not act as
@@ -620,7 +610,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
kernel's I/O cache, and might therefore still be fetched without
requiring a physical read. Users interested in obtaining more
detailed information on <productname>PostgreSQL</productname> I/O behavior are
- advised to use the <productname>PostgreSQL</productname> statistics collector
+ advised to use the <productname>PostgreSQL</productname> activity statistics
in combination with operating system utilities that allow insight
into the kernel's handling of I/O.
</para>
@@ -1057,10 +1047,6 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry><literal>LogicalLauncherMain</literal></entry>
<entry>Waiting in main loop of logical replication launcher process.</entry>
</row>
- <row>
- <entry><literal>PgStatMain</literal></entry>
- <entry>Waiting in main loop of statistics collector process.</entry>
- </row>
<row>
<entry><literal>RecoveryWalStream</literal></entry>
<entry>Waiting in main loop of startup process for WAL to arrive, during
@@ -1815,6 +1801,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</thead>
<tbody>
+ <row>
+ <entry><literal>ActivityStatistics</literal></entry>
+ <entry>Waiting to write out activity statistics to shared memory.</entry>
+ </row>
<row>
<entry><literal>AddinShmemInit</literal></entry>
<entry>Waiting to manage an extension's space allocation in shared
@@ -5738,9 +5728,10 @@ SELECT pg_stat_get_backend_pid(s.backendid) AS pid,
<entry><literal>performing final cleanup</literal></entry>
<entry>
<command>VACUUM</command> is performing final cleanup. During this phase,
- <command>VACUUM</command> will vacuum the free space map, update statistics
- in <literal>pg_class</literal>, and report statistics to the statistics
- collector. When this phase is completed, <command>VACUUM</command> will end.
+ <command>VACUUM</command> will vacuum the free space map, update
+ statistics in <literal>pg_class</literal>, and system-wide activity
+ statistics. When this phase is completed, <command>VACUUM</command>
+ will end.
</entry>
</row>
</tbody>
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index 0b2e2de87b..ad88efdfee 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -1288,11 +1288,10 @@ PostgreSQL documentation
</para>
<para>
- The database activity of <application>pg_dump</application> is
- normally collected by the statistics collector. If this is
- undesirable, you can set parameter <varname>track_counts</varname>
- to false via <envar>PGOPTIONS</envar> or the <literal>ALTER
- USER</literal> command.
+ The database activity of <application>pg_dump</application> is normally
+ collected. If this is undesirable, you can set
+ parameter <varname>track_counts</varname> to false
+ via <envar>PGOPTIONS</envar> or the <literal>ALTER USER</literal> command.
</para>
</refsect1>
--
2.18.4
v36-0007-Remove-the-GUC-stats_temp_directory.patch (text/x-patch, 9.6K) download
From 980614004a05151ca57eabff61c787300a27ff8b Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Fri, 13 Mar 2020 17:00:44 +0900
Subject: [PATCH v36 7/7] Remove the GUC stats_temp_directory
The GUC used to specify the directory to store temporary statistics
files. It is no longer needed by the stats collector but still used by
the programs in bin and contrib, and maybe other extensions. Thus this
patch removes the GUC but some backing variables and macro definitions
are left alone for backward compatibility.
---
doc/src/sgml/backup.sgml | 2 -
doc/src/sgml/config.sgml | 19 ---------
doc/src/sgml/storage.sgml | 3 +-
src/backend/postmaster/pgstat.c | 13 +++---
src/backend/replication/basebackup.c | 13 ++----
src/backend/utils/misc/guc.c | 41 -------------------
src/backend/utils/misc/postgresql.conf.sample | 1 -
src/include/pgstat.h | 5 ++-
src/test/perl/PostgresNode.pm | 4 --
9 files changed, 13 insertions(+), 88 deletions(-)
diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml
index b9331830f7..5096963234 100644
--- a/doc/src/sgml/backup.sgml
+++ b/doc/src/sgml/backup.sgml
@@ -1146,8 +1146,6 @@ SELECT pg_stop_backup();
<filename>pg_snapshots/</filename>, <filename>pg_stat_tmp/</filename>,
and <filename>pg_subtrans/</filename> (but not the directories themselves) can be
omitted from the backup as they will be initialized on postmaster startup.
- If <xref linkend="guc-stats-temp-directory"/> is set and is under the data
- directory then the contents of that directory can also be omitted.
</para>
<para>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 530f41c194..7f2f18f294 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -7303,25 +7303,6 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
- <varlistentry id="guc-stats-temp-directory" xreflabel="stats_temp_directory">
- <term><varname>stats_temp_directory</varname> (<type>string</type>)
- <indexterm>
- <primary><varname>stats_temp_directory</varname> configuration parameter</primary>
- </indexterm>
- </term>
- <listitem>
- <para>
- Sets the directory to store temporary statistics data in. This can be
- a path relative to the data directory or an absolute path. The default
- is <filename>pg_stat_tmp</filename>. Pointing this at a RAM-based
- file system will decrease physical I/O requirements and can lead to
- improved performance.
- This parameter can only be set in the <filename>postgresql.conf</filename>
- file or on the server command line.
- </para>
- </listitem>
- </varlistentry>
-
</variablelist>
</sect2>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 3234adb639..559f75fb54 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -122,8 +122,7 @@ Item
<row>
<entry><filename>pg_stat_tmp</filename></entry>
- <entry>Subdirectory containing temporary files for the statistics
- subsystem</entry>
+ <entry>Subdirectory containing ephemeral files for extensions</entry>
</row>
<row>
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 90c4ea31a0..b07add0a4f 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -96,15 +96,12 @@ bool pgstat_track_counts = false;
int pgstat_track_functions = TRACK_FUNC_OFF;
int pgstat_track_activity_query_size = 1024;
-/* ----------
- * Built from GUC parameter
- * ----------
+/*
+ * This used to be a GUC variable and is no longer used in this file, but left
+ * alone just for backward compatibility for extensions, having the default
+ * value.
*/
-char *pgstat_stat_directory = NULL;
-
-/* No longer used, but will be removed with GUC */
-char *pgstat_stat_filename = NULL;
-char *pgstat_stat_tmpname = NULL;
+char *pgstat_stat_directory = PG_STAT_TMP_DIR;
typedef struct StatsShmemStruct
{
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index cc6b2bb5de..357008915d 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -261,7 +261,6 @@ perform_base_backup(basebackup_options *opt)
StringInfo labelfile;
StringInfo tblspc_map_file;
backup_manifest_info manifest;
- int datadirpathlen;
List *tablespaces = NIL;
backup_total = 0;
@@ -284,8 +283,6 @@ perform_base_backup(basebackup_options *opt)
Assert(CurrentResourceOwner == NULL);
CurrentResourceOwner = ResourceOwnerCreate(NULL, "base backup");
- datadirpathlen = strlen(DataDir);
-
backup_started_in_recovery = RecoveryInProgress();
labelfile = makeStringInfo();
@@ -318,13 +315,9 @@ perform_base_backup(basebackup_options *opt)
* Calculate the relative path of temporary statistics directory in
* order to skip the files which are located in that directory later.
*/
- if (is_absolute_path(pgstat_stat_directory) &&
- strncmp(pgstat_stat_directory, DataDir, datadirpathlen) == 0)
- statrelpath = psprintf("./%s", pgstat_stat_directory + datadirpathlen + 1);
- else if (strncmp(pgstat_stat_directory, "./", 2) != 0)
- statrelpath = psprintf("./%s", pgstat_stat_directory);
- else
- statrelpath = pgstat_stat_directory;
+
+ Assert(strchr(PG_STAT_TMP_DIR, '/') == NULL);
+ statrelpath = psprintf("./%s", PG_STAT_TMP_DIR);
/* Add a node for the base directory at the end */
ti = palloc0(sizeof(tablespaceinfo));
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 36113378f7..70c573dcaa 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -202,7 +202,6 @@ static bool check_autovacuum_work_mem(int *newval, void **extra, GucSource sourc
static bool check_effective_io_concurrency(int *newval, void **extra, GucSource source);
static bool check_maintenance_io_concurrency(int *newval, void **extra, GucSource source);
static bool check_huge_page_size(int *newval, void **extra, GucSource source);
-static void assign_pgstat_temp_directory(const char *newval, void *extra);
static bool check_application_name(char **newval, void **extra, GucSource source);
static void assign_application_name(const char *newval, void *extra);
static bool check_cluster_name(char **newval, void **extra, GucSource source);
@@ -4309,17 +4308,6 @@ static struct config_string ConfigureNamesString[] =
NULL, NULL, NULL
},
- {
- {"stats_temp_directory", PGC_SIGHUP, STATS_ACTIVITY,
- gettext_noop("Writes temporary statistics files to the specified directory."),
- NULL,
- GUC_SUPERUSER_ONLY
- },
- &pgstat_temp_directory,
- PG_STAT_TMP_DIR,
- check_canonical_path, assign_pgstat_temp_directory, NULL
- },
-
{
{"synchronous_standby_names", PGC_SIGHUP, REPLICATION_PRIMARY,
gettext_noop("Number of synchronous standbys and list of names of potential synchronous ones."),
@@ -11608,35 +11596,6 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
return true;
}
-static void
-assign_pgstat_temp_directory(const char *newval, void *extra)
-{
- /* check_canonical_path already canonicalized newval for us */
- char *dname;
- char *tname;
- char *fname;
-
- /* directory */
- dname = guc_malloc(ERROR, strlen(newval) + 1); /* runtime dir */
- sprintf(dname, "%s", newval);
-
- /* global stats */
- tname = guc_malloc(ERROR, strlen(newval) + 12); /* /global.tmp */
- sprintf(tname, "%s/global.tmp", newval);
- fname = guc_malloc(ERROR, strlen(newval) + 13); /* /global.stat */
- sprintf(fname, "%s/global.stat", newval);
-
- if (pgstat_stat_directory)
- free(pgstat_stat_directory);
- pgstat_stat_directory = dname;
- if (pgstat_stat_tmpname)
- free(pgstat_stat_tmpname);
- pgstat_stat_tmpname = tname;
- if (pgstat_stat_filename)
- free(pgstat_stat_filename);
- pgstat_stat_filename = fname;
-}
-
static bool
check_application_name(char **newval, void **extra, GucSource source)
{
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 668a2d033a..7183c08305 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -586,7 +586,6 @@
#track_io_timing = off
#track_functions = none # none, pl, all
#track_activity_query_size = 1024 # (change requires restart)
-#stats_temp_directory = 'pg_stat_tmp'
# - Monitoring -
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 29a8737498..9fa87de887 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -32,7 +32,10 @@
#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
-/* Default directory to store temporary statistics data in */
+/*
+ * This used to be the directory to store temporary statistics data in but is
+ * no longer used. Defined here for backward compatibility.
+ */
#define PG_STAT_TMP_DIR "pg_stat_tmp"
/* Values for track_functions GUC variable --- order is significant! */
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1488bffa2b..bb5474b878 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -455,10 +455,6 @@ sub init
print $conf TestLib::slurp_file($ENV{TEMP_CONFIG})
if defined $ENV{TEMP_CONFIG};
- # XXX Neutralize any stats_temp_directory in TEMP_CONFIG. Nodes running
- # concurrently must not share a stats_temp_directory.
- print $conf "stats_temp_directory = 'pg_stat_tmp'\n";
-
if ($params{allows_streaming})
{
if ($params{allows_streaming} eq "logical")
--
2.18.4
* Re: shared-memory based stats collector
2020-09-03 17:16 Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
2020-09-08 08:01 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2020-09-08 08:55 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
@ 2020-09-09 08:01 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 0 replies; 17+ messages in thread
From: Kyotaro Horiguchi @ 2020-09-09 08:01 UTC (permalink / raw)
To: sfrost@snowman.net; +Cc: alvherre@2ndquadrant.com; andres@anarazel.de; michael@paquier.xyz; thomas.munro@gmail.com; tomas.vondra@2ndquadrant.com; ah@cybertec.at; magnus@hagander.net; robertmhaas@gmail.com; tgl@sss.pgh.pa.us; pgsql-hackers
At Tue, 08 Sep 2020 17:55:57 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> At Tue, 08 Sep 2020 17:01:47 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > At Thu, 3 Sep 2020 13:16:59 -0400, Stephen Frost <sfrost@snowman.net> wrote in> > I've looked through (some of) this thread and through the patches also
> > > and hope to provide a review of the bits that should be targetting v14
> > > (unlike the above) soon.
> >
> > Thanks. Current the patch is found to lead to write contention heavier
> > than the current stats collector when nearly a thousand backend
> > heavily write to the same table. I need to address that.
> >
> > - Collect stats via sockets (in the same way as the current implement)
> > and write/read to/from shared memory.
> >
> > - Use our own lock-free (maybe) ring-buffer before stats-write enters
> > lock-waiting mode, then new stats collector(!) process consumes the
> > queue.
> >
> > - Some other measures..
- Make dshash search less frequent. To find the actual source of the
contension, We're going to measure performance with the attached on
top of the latest patch let sessions cache the result of dshash
searches for the session lifetime. (table-dropping vacuum clears the
local hash.)
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachments:
v36-0008-Experimental-local-cache-for-dshash.patch.txt (text/plain, 4.6K) download
From 0ce7b7a891326a3fdcf85b001586f721bef569f9 Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
Date: Wed, 9 Sep 2020 16:44:57 +0900
Subject: [PATCH v36 8/8] Experimental local cache for dshash.
This patch suffers LWLock contension than unpatched while very many
sessions commits frequently. As an effort to identify the source, this
patch caches dshash information in a local hash to avoid dshash locks.
---
src/backend/postmaster/pgstat.c | 67 +++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index b07add0a4f..caa067e9fe 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -35,6 +35,7 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
+#include "common/hashfn.h"
#include "libpq/libpq.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -112,6 +113,8 @@ typedef struct StatsShmemStruct
dsa_pointer slru_stats; /* Ditto for SLRU stats */
int refcount; /* # of processes that is attaching the shared
* stats memory */
+ pg_atomic_uint64 del_ent_count; /* # of entries deleted. not protected by
+ StatsLock */
} StatsShmemStruct;
/* BgWriter global statistics counters */
@@ -254,6 +257,7 @@ typedef struct PgStatSharedSLRUStats
typedef struct PgStatLocalHashEntry
{
PgStatHashEntryKey key; /* hash key */
+ char status; /* for simplehash use */
PgStatEnvelope *env; /* pointer to stats envelope in heap */
} PgStatLocalHashEntry;
@@ -280,6 +284,18 @@ static const dshash_parameters dsh_rootparams = {
LWTRANCHE_STATS
};
+/* define hashtable for dshash caching */
+#define SH_PREFIX pgstat_cache
+#define SH_ELEMENT_TYPE PgStatLocalHashEntry
+#define SH_KEY_TYPE PgStatHashEntryKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) hash_bytes((unsigned char *)&key, sizeof(PgStatHashEntryKey))
+#define SH_EQUAL(tb, a, b) (memcmp(&a, &b, sizeof(PgStatHashEntryKey)) == 0)
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
/* The shared hash to index activity stats entries. */
static dshash_table *pgStatSharedHash = NULL;
@@ -326,6 +342,8 @@ typedef struct TwoPhasePgStatRecord
} TwoPhasePgStatRecord;
/* Variables for backend status snapshot */
+static pgstat_cache_hash *pgStatEntHash = NULL;
+static int pgStatEntHashAge = 0;
static MemoryContext pgStatLocalContext = NULL;
static MemoryContext pgStatSnapshotContext = NULL;
static HTAB *pgStatSnapshotHash = NULL;
@@ -437,6 +455,7 @@ StatsShmemInit(void)
Assert(!found);
StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ pg_atomic_init_u64(&StatsShmem->del_ent_count, 0);
}
}
@@ -1432,6 +1451,9 @@ pgstat_vacuum_stat(void)
delete_stat_entry(p->type, p->databaseid, p->objectid, true);
}
+
+ if (nvictims > 0)
+ pg_atomic_add_fetch_u64(&StatsShmem->del_ent_count, 1);
}
@@ -1493,6 +1515,8 @@ pgstat_drop_database(Oid databaseid)
delete_stat_entry((*p)->type, (*p)->databaseid, (*p)->objectid, true);
pfree(envlist);
+
+ pg_atomic_add_fetch_u64(&StatsShmem->del_ent_count, 1);
}
@@ -5119,6 +5143,7 @@ get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
{
bool create = (initfunc != NULL);
PgStatHashEntry *shent;
+ PgStatLocalHashEntry *loent;
PgStatEnvelope *shenv = NULL;
PgStatHashEntryKey key;
bool myfound;
@@ -5128,6 +5153,31 @@ get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
key.type = type;
key.databaseid = dbid;
key.objectid = objid;
+
+ if (pgStatEntHash)
+ {
+ uint64 currage;
+
+ currage = pg_atomic_read_u64(&StatsShmem->del_ent_count);
+
+ if (currage == pgStatEntHashAge)
+ {
+ loent = pgstat_cache_lookup(pgStatEntHash, key);
+
+ if (loent)
+ {
+ if (found)
+ *found = true;
+ return loent->env;
+ }
+ }
+ else
+ {
+ pgstat_cache_destroy(pgStatEntHash);
+ pgStatEntHash = NULL;
+ }
+ }
+
shent = dshash_find_extended(pgStatSharedHash, &key,
create, nowait, create, &myfound);
if (shent)
@@ -5159,6 +5209,23 @@ get_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
shenv = dsa_get_address(area, shent->env);
dshash_release_lock(pgStatSharedHash, shent);
+
+ /* register to local hash if possible */
+ if (pgStatEntHash || CacheMemoryContext)
+ {
+ if (pgStatEntHash == NULL)
+ {
+ pgStatEntHash =
+ pgstat_cache_create(CacheMemoryContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
+ pgStatEntHashAge =
+ pg_atomic_read_u64(&StatsShmem->del_ent_count);
+ }
+
+ loent = pgstat_cache_insert(pgStatEntHash, key, &myfound);
+ Assert(!myfound);
+ loent->env = shenv;
+ }
}
if (found)
--
2.18.4
end of thread, other threads:[~2020-09-09 08:01 UTC | newest]
Thread overview: 17+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-11-12 08:26 [PATCH 4/5] Shared-memory based stats collector Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
2018-11-12 08:26 [PATCH 4/7] Shared-memory based stats collector Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
2019-02-21 03:44 [PATCH v24 4/5] Shared-memory based stats collector Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
2019-02-21 03:44 [PATCH 4/5] Shared-memory based stats collector Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
2020-03-19 06:11 [PATCH v45 4/7] Shared-memory based stats collector Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
2020-03-19 06:11 [PATCH v46 4/7] Shared-memory based stats collector Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
2020-03-19 06:11 [PATCH v35 5/7] Shared-memory based stats collector Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
2020-03-19 06:11 [PATCH v29 5/7] Shared-memory based stats collector Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
2020-03-19 06:11 [PATCH v30 5/7] Shared-memory based stats collector Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
2020-03-19 06:11 [PATCH v25 6/8] Shared-memory based stats collector Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
2020-03-19 06:11 [PATCH v27 5/7] Shared-memory based stats collector Kyotaro Horiguchi <horikyoga.ntt@gmail.com>
2020-09-03 17:16 Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
2020-09-07 02:04 ` Re: shared-memory based stats collector Michael Paquier <michael@paquier.xyz>
2020-09-07 13:46 ` Re: shared-memory based stats collector Stephen Frost <sfrost@snowman.net>
2020-09-08 08:01 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2020-09-08 08:55 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2020-09-09 08:01 ` Re: shared-memory based stats collector Kyotaro Horiguchi <horikyota.ntt@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox