public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v45 4/7] Shared-memory based stats collector
124+ messages / 27 participants
[nested] [flat]
* [PATCH v45 4/7] Shared-memory based stats collector
@ 2020-03-19 06:11 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
Previously activity statistics is collected via sockets and shared
among backends through files periodically. Such files reaches tens of
megabytes and are created at most every 1 second and such large data
is serialized by stats collector then de-serialized on every backend
periodically. To evade that large cost, this patch places activity
statistics data on shared memory. Each backend accumulates statistics
numbers locally then tries to move them onto the shared statistics at
every transaction end but with intervals not shorter than 10 seconds.
Until 60 second has elapsed since the last flushing to shared stats,
lock failure postpones stats flushing to try to alleviate lock
contention that slows down transactions. Finally stats flush waits
for locks so that shared statistics doesn't get stale.
---
src/backend/access/heap/heapam_handler.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 2 +-
src/backend/access/transam/xlog.c | 6 +-
src/backend/catalog/index.c | 24 +-
src/backend/commands/analyze.c | 8 +-
src/backend/commands/dbcommands.c | 2 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/vacuum.c | 4 +-
src/backend/postmaster/autovacuum.c | 76 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 26 +-
src/backend/postmaster/pgarch.c | 12 +-
src/backend/postmaster/pgstat.c | 6135 +++++++----------
src/backend/postmaster/postmaster.c | 88 +-
src/backend/replication/basebackup.c | 4 +-
src/backend/replication/slot.c | 12 +-
src/backend/storage/buffer/bufmgr.c | 8 +-
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/lmgr/lwlock.c | 4 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/smgr.c | 4 +-
src/backend/tcop/postgres.c | 41 +-
src/backend/utils/adt/pgstatfuncs.c | 95 +-
src/backend/utils/cache/relcache.c | 5 +
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 3 -
src/backend/utils/init/postinit.c | 11 +
src/backend/utils/misc/guc.c | 14 +-
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/include/miscadmin.h | 3 +-
src/include/pgstat.h | 727 +-
src/include/storage/lwlock.h | 1 +
src/include/utils/guc_tables.h | 2 +-
src/include/utils/timeout.h | 1 +
35 files changed, 2791 insertions(+), 4553 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 10ddde4ecf..98df20ce96 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1086,8 +1086,8 @@ heapam_scan_analyze_next_tuple(TableScanDesc scan, TransactionId OldestXmin,
* our own. In this case we should count and sample the row,
* to accommodate users who load a table and analyze it in one
* transaction. (pgstat_report_analyze has to adjust the
- * numbers we send to the stats collector to make this come
- * out right.)
+ * numbers we report to the activity stats facility to make
+ * this come out right.)
*/
if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(targtuple->t_data)))
{
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f3d2265fad..ceccb7e19e 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -598,7 +598,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
new_min_multi,
false);
- /* report results to the stats collector, too */
+ /* report results to the activity stats facility, too */
pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
Max(new_live_tuples, 0),
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ede93ad7fd..5d775ba84c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2207,7 +2207,7 @@ AdvanceXLInsertBuffer(XLogRecPtr upto, bool opportunistic)
WriteRqst.Flush = 0;
XLogWrite(WriteRqst, false);
LWLockRelease(WALWriteLock);
- WalStats.m_wal_buffers_full++;
+ WalStats.wal_buffers_full++;
TRACE_POSTGRESQL_WAL_BUFFER_WRITE_DIRTY_DONE();
}
/* Re-acquire WALBufMappingLock and retry */
@@ -8592,8 +8592,8 @@ LogCheckpointEnd(bool restartpoint)
CheckpointStats.ckpt_sync_end_t);
/* Accumulate checkpoint timing summary data, in milliseconds. */
- BgWriterStats.m_checkpoint_write_time += write_msecs;
- BgWriterStats.m_checkpoint_sync_time += sync_msecs;
+ CheckPointerStats.checkpoint_write_time += write_msecs;
+ CheckPointerStats.checkpoint_sync_time += sync_msecs;
/*
* All of the published timing statistics are accounted for. Only
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cffbc0ac38..3f55c34909 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1854,28 +1854,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Copy over statistics from old to new index
+ * The data will be flushed by the next pgstat_report_stat()
+ * call.
*/
- {
- PgStat_StatTabEntry *tabentry;
-
- tabentry = pgstat_fetch_stat_tabentry(oldIndexId);
- if (tabentry)
- {
- if (newClassRel->pgstat_info)
- {
- newClassRel->pgstat_info->t_counts.t_numscans = tabentry->numscans;
- newClassRel->pgstat_info->t_counts.t_tuples_returned = tabentry->tuples_returned;
- newClassRel->pgstat_info->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
- newClassRel->pgstat_info->t_counts.t_blocks_hit = tabentry->blocks_hit;
-
- /*
- * The data will be sent by the next pgstat_report_stat()
- * call.
- */
- }
- }
- }
+ pgstat_copy_index_counters(oldIndexId, newClassRel->pgstat_info);
/* Copy data of pg_statistic from the old index to the new one */
CopyStatistics(oldIndexId, newIndexId);
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 7295cf0215..308b4ab034 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -644,10 +644,10 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
}
/*
- * Report ANALYZE to the stats collector, too. However, if doing
- * inherited stats we shouldn't report, because the stats collector only
- * tracks per-table stats. Reset the changes_since_analyze counter only
- * if we analyzed all columns; otherwise, there is still work for
+ * Report ANALYZE to the activity stats facility, too. However, if doing
+ * inherited stats we shouldn't report, because the activity stats facility
+ * only tracks per-table stats. Reset the changes_since_analyze counter
+ * only if we analyzed all columns; otherwise, there is still work for
* auto-analyze to do.
*/
if (!inh)
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 2b159b60eb..acea4de382 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -971,7 +971,7 @@ dropdb(const char *dbname, bool missing_ok, bool force)
DropDatabaseBuffers(db_id);
/*
- * Tell the stats collector to forget it immediately, too.
+ * Tell the active stats facility to forget it immediately, too.
*/
pgstat_drop_database(db_id);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..1464b97c7f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -336,10 +336,10 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
/*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
+ * Inform activity stats facility about our activity: basically, we
+ * truncated the matview and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
*/
pgstat_count_truncate(matviewRel);
if (!stmt->skipData)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index b97d48ee01..e80ab5edd0 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -319,8 +319,8 @@ vacuum(List *relations, VacuumParams *params,
errmsg("VACUUM option DISABLE_PAGE_SKIPPING cannot be used with FULL")));
/*
- * Send info about dead objects to the statistics collector, unless we are
- * in autovacuum --- autovacuum.c does this for itself.
+ * Send info about dead objects to the activity statistics facility, unless
+ * we are in autovacuum --- autovacuum.c does this for itself.
*/
if ((params->options & VACOPT_VACUUM) && !IsAutoVacuumWorkerProcess())
pgstat_vacuum_stat();
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 47e60ca561..194a02024b 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -342,9 +342,6 @@ static void autovacuum_do_vac_analyze(autovac_table *tab,
BufferAccessStrategy bstrategy);
static AutoVacOpts *extract_autovac_opts(HeapTuple tup,
TupleDesc pg_class_desc);
-static PgStat_StatTabEntry *get_pgstat_tabentry_relid(Oid relid, bool isshared,
- PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry);
static void perform_work_item(AutoVacuumWorkItem *workitem);
static void autovac_report_activity(autovac_table *tab);
static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
@@ -1684,12 +1681,12 @@ AutoVacWorkerMain(int argc, char *argv[])
char dbname[NAMEDATALEN];
/*
- * Report autovac startup to the stats collector. We deliberately do
- * this before InitPostgres, so that the last_autovac_time will get
- * updated even if the connection attempt fails. This is to prevent
- * autovac from getting "stuck" repeatedly selecting an unopenable
- * database, rather than making any progress on stuff it can connect
- * to.
+ * Report autovac startup to the activity stats facility. We
+ * deliberately do this before InitPostgres, so that the
+ * last_autovac_time will get updated even if the connection attempt
+ * fails. This is to prevent autovac from getting "stuck" repeatedly
+ * selecting an unopenable database, rather than making any progress on
+ * stuff it can connect to.
*/
pgstat_report_autovac(dbid);
@@ -1961,8 +1958,6 @@ do_autovacuum(void)
HASHCTL ctl;
HTAB *table_toast_map;
ListCell *volatile cell;
- PgStat_StatDBEntry *shared;
- PgStat_StatDBEntry *dbentry;
BufferAccessStrategy bstrategy;
ScanKeyData key;
TupleDesc pg_class_desc;
@@ -1981,17 +1976,11 @@ do_autovacuum(void)
ALLOCSET_DEFAULT_SIZES);
MemoryContextSwitchTo(AutovacMemCxt);
- /*
- * may be NULL if we couldn't find an entry (only happens if we are
- * forcing a vacuum for anti-wrap purposes).
- */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
-
/* Start a transaction so our commands have one to play into. */
StartTransactionCommand();
/*
- * Clean up any dead statistics collector entries for this DB. We always
+ * Clean up any dead activity statistics entries for this DB. We always
* want to do this exactly once per DB-processing cycle, even if we find
* nothing worth vacuuming in the database.
*/
@@ -2034,9 +2023,6 @@ do_autovacuum(void)
/* StartTransactionCommand changed elsewhere */
MemoryContextSwitchTo(AutovacMemCxt);
- /* The database hash where pgstat keeps shared relations */
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
-
classRel = table_open(RelationRelationId, AccessShareLock);
/* create a copy so we can use it after closing pg_class */
@@ -2114,8 +2100,8 @@ do_autovacuum(void)
/* Fetch reloptions and the pgstat entry for this table */
relopts = extract_autovac_opts(tuple, pg_class_desc);
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
/* Check if it needs vacuum or analyze */
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
@@ -2198,8 +2184,8 @@ do_autovacuum(void)
}
/* Fetch the pgstat entry for this table */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, relopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -2758,29 +2744,6 @@ extract_autovac_opts(HeapTuple tup, TupleDesc pg_class_desc)
return av;
}
-/*
- * get_pgstat_tabentry_relid
- *
- * Fetch the pgstat entry of a table, either local to a database or shared.
- */
-static PgStat_StatTabEntry *
-get_pgstat_tabentry_relid(Oid relid, bool isshared, PgStat_StatDBEntry *shared,
- PgStat_StatDBEntry *dbentry)
-{
- PgStat_StatTabEntry *tabentry = NULL;
-
- if (isshared)
- {
- if (PointerIsValid(shared))
- tabentry = hash_search(shared->tables, &relid,
- HASH_FIND, NULL);
- }
- else if (PointerIsValid(dbentry))
- tabentry = hash_search(dbentry->tables, &relid,
- HASH_FIND, NULL);
-
- return tabentry;
-}
/*
* table_recheck_autovac
@@ -2984,17 +2947,10 @@ recheck_relation_needs_vacanalyze(Oid relid,
bool *wraparound)
{
PgStat_StatTabEntry *tabentry;
- PgStat_StatDBEntry *shared = NULL;
- PgStat_StatDBEntry *dbentry = NULL;
-
- if (classForm->relisshared)
- shared = pgstat_fetch_stat_dbentry(InvalidOid);
- else
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
/* fetch the pgstat table entry */
- tabentry = get_pgstat_tabentry_relid(relid, classForm->relisshared,
- shared, dbentry);
+ tabentry = pgstat_fetch_stat_tabentry_extended(classForm->relisshared,
+ relid);
relation_needs_vacanalyze(relid, avopts, classForm, tabentry,
effective_multixact_freeze_max_age,
@@ -3024,7 +2980,7 @@ recheck_relation_needs_vacanalyze(Oid relid,
*
* For analyze, the analysis done is that the number of tuples inserted,
* deleted and updated since the last analyze exceeds a threshold calculated
- * in the same fashion as above. Note that the collector actually stores
+ * in the same fashion as above. Note that the activity statistics stores
* the number of tuples (both live and dead) that there were as of the last
* analyze. This is asymmetric to the VACUUM case.
*
@@ -3034,8 +2990,8 @@ recheck_relation_needs_vacanalyze(Oid relid,
*
* A table whose autovacuum_enabled option is false is
* automatically skipped (unless we have to vacuum it due to freeze_max_age).
- * Thus autovacuum can be disabled for specific tables. Also, when the stats
- * collector does not have data about a table, it will be skipped.
+ * Thus autovacuum can be disabled for specific tables. Also, when the activity
+ * statistics does not have data about a table, it will be skipped.
*
* A table whose vac_base_thresh value is < 0 takes the base value from the
* autovacuum_vacuum_threshold GUC variable. Similarly, a vac_scale_factor
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 715d5195bb..679992dc89 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -244,9 +244,9 @@ BackgroundWriterMain(void)
can_hibernate = BgBufferSync(&wb_context);
/*
- * Send off activity statistics to the stats collector
+ * Send off activity statistics to the activity stats facility
*/
- pgstat_send_bgwriter();
+ pgstat_report_bgwriter();
if (FirstCallSinceLastCheckpoint())
{
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 54a818bf61..309a9997e1 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -360,7 +360,7 @@ CheckpointerMain(void)
if (((volatile CheckpointerShmemStruct *) CheckpointerShmem)->ckpt_flags)
{
do_checkpoint = true;
- BgWriterStats.m_requested_checkpoints++;
+ CheckPointerStats.requested_checkpoints++;
}
/*
@@ -374,7 +374,7 @@ CheckpointerMain(void)
if (elapsed_secs >= CheckPointTimeout)
{
if (!do_checkpoint)
- BgWriterStats.m_timed_checkpoints++;
+ CheckPointerStats.timed_checkpoints++;
do_checkpoint = true;
flags |= CHECKPOINT_CAUSE_TIME;
}
@@ -495,17 +495,11 @@ CheckpointerMain(void)
/* Check for archive_timeout and switch xlog files if necessary. */
CheckArchiveTimeout();
- /*
- * Send off activity statistics to the stats collector. (The reason
- * why we re-use bgwriter-related code for this is that the bgwriter
- * and checkpointer used to be just one process. It's probably not
- * worth the trouble to split the stats support into two independent
- * stats message types.)
- */
- pgstat_send_bgwriter();
+ /* Send off activity statistics to the activity stats facility. */
+ pgstat_report_checkpointer();
/* Send WAL statistics to the stats collector. */
- pgstat_send_wal();
+ pgstat_report_wal();
/*
* If any checkpoint flags have been set, redo the loop to handle the
@@ -711,9 +705,9 @@ CheckpointWriteDelay(int flags, double progress)
CheckArchiveTimeout();
/*
- * Report interim activity statistics to the stats collector.
+ * Report interim activity statistics.
*/
- pgstat_send_bgwriter();
+ pgstat_report_checkpointer();
/*
* This sleep used to be connected to bgwriter_delay, typically 200ms.
@@ -1257,8 +1251,10 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
/* Transfer stats counts into pending pgstats message */
- BgWriterStats.m_buf_written_backend += CheckpointerShmem->num_backend_writes;
- BgWriterStats.m_buf_fsync_backend += CheckpointerShmem->num_backend_fsync;
+ CheckPointerStats.buf_written_backend
+ += CheckpointerShmem->num_backend_writes;
+ CheckPointerStats.buf_fsync_backend
+ += CheckpointerShmem->num_backend_fsync;
CheckpointerShmem->num_backend_writes = 0;
CheckpointerShmem->num_backend_fsync = 0;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 9a2e21bf86..504443dcc0 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -372,20 +372,20 @@ pgarch_ArchiverCopyLoop(void)
pgarch_archiveDone(xlog);
/*
- * Tell the collector about the WAL file that we successfully
- * archived
+ * Tell the activity statistics facility about the WAL file
+ * that we successfully archived
*/
- pgstat_send_archiver(xlog, false);
+ pgstat_report_archiver(xlog, false);
break; /* out of inner retry loop */
}
else
{
/*
- * Tell the collector about the WAL file that we failed to
- * archive
+ * Tell the activity statistics facility about the WAL file
+ * that we failed to archive
*/
- pgstat_send_archiver(xlog, true);
+ pgstat_report_archiver(xlog, true);
if (++failures >= NUM_ARCHIVE_RETRIES)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 3f24a33ef1..ecf9d9adcc 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1,15 +1,22 @@
/* ----------
* pgstat.c
*
- * All the statistics collector stuff hacked up in one big, ugly file.
+ * Activity Statistics facility.
*
- * TODO: - Separate collector, postmaster and backend stuff
- * into different files.
+ * Collects activity statistics, e.g. per-table access statistics, of
+ * all backends in shared memory. The activity numbers are first stored
+ * locally in each process, then flushed to shared memory at commit
+ * time or by idle-timeout.
*
- * - Add some automatic call for pgstat vacuuming.
+ * To avoid congestion on the shared memory, shared stats is updated no more
+ * often than once per PGSTAT_MIN_INTERVAL (10000ms). If some local numbers
+ * remain unflushed for lock failure, retry with intervals that is initially
+ * PGSTAT_RETRY_MIN_INTERVAL (1000ms) then doubled at every retry. Finally we
+ * force update after PGSTAT_MAX_INTERVAL (60000ms) since the first trial.
*
- * - Add a pgstat config column to pg_database, so this
- * entire thing can be enabled/disabled on a per db basis.
+ * The first process that uses activity statistics facility creates the area
+ * then load the stored stats file if any, and the last process at shutdown
+ * writes the shared stats to the file then destroy the area before exit.
*
* Copyright (c) 2001-2021, PostgreSQL Global Development Group
*
@@ -19,18 +26,6 @@
#include "postgres.h"
#include <unistd.h>
-#include <fcntl.h>
-#include <sys/param.h>
-#include <sys/time.h>
-#include <sys/socket.h>
-#include <netdb.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <signal.h>
-#include <time.h>
-#ifdef HAVE_SYS_SELECT_H
-#include <sys/select.h>
-#endif
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -40,13 +35,9 @@
#include "access/xact.h"
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
-#include "common/ip.h"
-#include "executor/instrument.h"
+#include "common/hashfn.h"
#include "libpq/libpq.h"
-#include "libpq/pqsignal.h"
-#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/fork_process.h"
@@ -54,20 +45,16 @@
#include "postmaster/postmaster.h"
#include "replication/slot.h"
#include "replication/walsender.h"
-#include "storage/backendid.h"
-#include "storage/dsm.h"
-#include "storage/fd.h"
+#include "storage/condition_variable.h"
#include "storage/ipc.h"
-#include "storage/latch.h"
#include "storage/lmgr.h"
-#include "storage/pg_shmem.h"
+#include "storage/proc.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/ps_status.h"
-#include "utils/rel.h"
+#include "utils/probes.h"
#include "utils/snapmgr.h"
#include "utils/timestamp.h"
@@ -75,35 +62,20 @@
* Timer definitions.
* ----------
*/
-#define PGSTAT_STAT_INTERVAL 500 /* Minimum time between stats file
- * updates; in milliseconds. */
+#define PGSTAT_MIN_INTERVAL 10000 /* Minimum interval of stats data
+ * updates; in milliseconds. */
-#define PGSTAT_RETRY_DELAY 10 /* How long to wait between checks for a
- * new file; in milliseconds. */
-
-#define PGSTAT_MAX_WAIT_TIME 10000 /* Maximum time to wait for a stats
- * file update; in milliseconds. */
-
-#define PGSTAT_INQ_INTERVAL 640 /* How often to ping the collector for a
- * new file; in milliseconds. */
-
-#define PGSTAT_RESTART_INTERVAL 60 /* How often to attempt to restart a
- * failed statistics collector; in
- * seconds. */
-
-#define PGSTAT_POLL_LOOP_COUNT (PGSTAT_MAX_WAIT_TIME / PGSTAT_RETRY_DELAY)
-#define PGSTAT_INQ_LOOP_COUNT (PGSTAT_INQ_INTERVAL / PGSTAT_RETRY_DELAY)
-
-/* Minimum receive buffer size for the collector's socket. */
-#define PGSTAT_MIN_RCVBUF (100 * 1024)
+#define PGSTAT_RETRY_MIN_INTERVAL 1000 /* Initial retry interval after
+ * PGSTAT_MIN_INTERVAL */
+#define PGSTAT_MAX_INTERVAL 60000 /* Longest interval of stats data
+ * updates */
/* ----------
- * The initial size hints for the hash tables used in the collector.
+ * The initial size hints for the hash tables used in the activity statistics.
* ----------
*/
-#define PGSTAT_DB_HASH_SIZE 16
-#define PGSTAT_TAB_HASH_SIZE 512
+#define PGSTAT_TABLE_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
@@ -118,7 +90,6 @@
*/
#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
-
/* ----------
* GUC parameters
* ----------
@@ -133,17 +104,11 @@ int pgstat_track_activity_query_size = 1024;
* ----------
*/
char *pgstat_stat_directory = NULL;
+
+/* No longer used, but will be removed with GUC */
char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
-/*
- * BgWriter and WAL global statistics counters.
- * Stored directly in a stats message structure so they can be sent
- * without needing to copy things around. We assume these init to zeroes.
- */
-PgStat_MsgBgWriter BgWriterStats;
-PgStat_MsgWal WalStats;
-
/*
* WAL usage counters saved from pgWALUsage at the previous call to
* pgstat_send_wal(). This is used to calculate how much WAL usage
@@ -170,73 +135,246 @@ static const char *const slru_names[] = {
#define SLRU_NUM_ELEMENTS lengthof(slru_names)
+/* struct for shared SLRU stats */
+typedef struct PgStatSharedSLRUStats
+{
+ PgStat_SLRUStats entry[SLRU_NUM_ELEMENTS];
+ LWLock lock;
+ pg_atomic_uint32 changecount;
+} PgStatSharedSLRUStats;
+
+StaticAssertDecl(sizeof(TimestampTz) == sizeof(pg_atomic_uint64),
+ "size of pg_atomic_uint64 doesn't match TimestampTz");
+
+typedef struct StatsShmemStruct
+{
+ dsa_handle stats_dsa_handle; /* handle for stats data area */
+ dshash_table_handle hash_handle; /* shared dbstat hash */
+ int refcount; /* # of processes that is attaching the
+ * shared stats memory */
+ /* Global stats structs */
+ PgStat_Archiver archiver_stats;
+ pg_atomic_uint32 archiver_changecount;
+ PgStat_BgWriter bgwriter_stats;
+ pg_atomic_uint32 bgwriter_changecount;
+ PgStat_CheckPointer checkpointer_stats;
+ pg_atomic_uint32 checkpointer_changecount;
+ PgStat_Wal wal_stats;
+ LWLock wal_stats_lock;
+ PgStatSharedSLRUStats slru_stats;
+ pg_atomic_uint32 slru_changecount;
+ pg_atomic_uint64 stats_timestamp;
+
+ /* Reset offsets, protected by StatsLock */
+ PgStat_Archiver archiver_reset_offset;
+ PgStat_BgWriter bgwriter_reset_offset;
+ PgStat_CheckPointer checkpointer_reset_offset;
+
+ /* file read/write protection */
+ bool attach_holdoff;
+ ConditionVariable holdoff_cv;
+
+ pg_atomic_uint64 gc_count; /* # of entries deleted. not
+ * protected by StatsLock */
+} StatsShmemStruct;
+
+/* BgWriter global statistics counters */
+PgStat_BgWriter BgWriterStats = {0};
+
+/* CheckPointer global statistics counters */
+PgStat_CheckPointer CheckPointerStats = {0};
+
+/* WAL global statistics counters */
+PgStat_Wal WalStats = {0} ;
+
/*
- * SLRU statistics counts waiting to be sent to the collector. These are
- * stored directly in stats message format so they can be sent without needing
- * to copy things around. We assume this variable inits to zeroes. Entries
- * are one-to-one with slru_names[].
+ * XXXX: always try to flush WAL stats. We don't want to manipulate another
+ * counter during XLogInsert so we don't have an effecient short cut to know
+ * whether any counter gets incremented.
*/
-static PgStat_MsgSLRU SLRUStats[SLRU_NUM_ELEMENTS];
+static inline bool
+walstats_pending(void)
+{
+ static const PgStat_Wal all_zeroes;
+
+ return memcmp(&WalStats, &all_zeroes,
+ offsetof(PgStat_Wal, stat_reset_timestamp)) != 0;
+}
+
+/*
+ * SLRU statistics counts waiting to be written to the shared activity
+ * statistics. We assume this variable inits to zeroes. Entries are
+ * one-to-one with slru_names[].
+ * Changes of SLRU counters are reported within critical sections so we use
+ * static memory in order to avoid memory allocation.
+ */
+static PgStat_SLRUStats local_SLRUStats[SLRU_NUM_ELEMENTS];
+static bool have_slrustats = false;
/* ----------
* Local data
* ----------
*/
-NON_EXEC_STATIC pgsocket pgStatSock = PGINVALID_SOCKET;
-
-static struct sockaddr_storage pgStatAddr;
-
-static time_t last_pgstat_start_time;
-
-static bool pgStatRunningInCollector = false;
+/* backend-lifetime storages */
+static StatsShmemStruct * StatsShmem = NULL;
+static dsa_area *area = NULL;
/*
- * Structures in which backends store per-table info that's waiting to be
- * sent to the collector.
- *
- * NOTE: once allocated, TabStatusArray structures are never moved or deleted
- * for the life of the backend. Also, we zero out the t_id fields of the
- * contained PgStat_TableStatus structs whenever they are not actively in use.
- * This allows relcache pgstat_info pointers to be treated as long-lived data,
- * avoiding repeated searches in pgstat_initstats() when a relation is
- * repeatedly opened during a transaction.
+ * Types to define shared statistics structure.
+ *
+ * Per-object statistics are stored in a "shared stats", corresponding struct
+ * that has a header part common among all object types in DSA-allocated
+ * memory. All shared stats are pointed from a dshash via a dsa_pointer. This
+ * structure make the shared stats immovable against dshash resizing, allows a
+ * backend point to shared stats entries via a native pointer and allows
+ * locking at stats-entry level. The per-entry locking reduces lock contention
+ * compared to partition lock of dshash. A backend accumulates stats numbers in
+ * a stats entry in the local memory space then flushes the numbers to shared
+ * stats entries at basically transaction end.
+ *
+ * Each stat entry type has a fixed member PgStat_HashEntryHeader as the first
+ * element.
+ *
+ * Shared stats are stored as:
+ *
+ * dshash pgStatSharedHash
+ * -> PgStatHashEntry (dshash entry)
+ * (dsa_pointer)-> PgStat_Stat*Entry (dsa memory block)
+ *
+ * Shared stats entries are directly pointed from pgstat_localhash hash:
+ *
+ * pgstat_localhash pgStatEntHash
+ * -> PgStatLocalHashEntry (equivalent of PgStatHashEntry)
+ * (native pointer)-> PgStat_Stat*Entry (dsa memory block)
+ *
+ * Local stats that are waiting for being flushed to share stats are stored as:
+ *
+ * pgstat_localhash pgStatLocalHash
+ * -> PgStatLocalHashEntry (local hash entry)
+ * (native pointer)-> PgStat_Stat*Entry/TableStatus (palloc'ed memory)
*/
-#define TABSTAT_QUANTUM 100 /* we alloc this many at a time */
-typedef struct TabStatusArray
+/* The types of statistics entries */
+typedef enum PgStatTypes
{
- struct TabStatusArray *tsa_next; /* link to next array, if any */
- int tsa_used; /* # entries currently used */
- PgStat_TableStatus tsa_entries[TABSTAT_QUANTUM]; /* per-table data */
-} TabStatusArray;
-
-static TabStatusArray *pgStatTabList = NULL;
+ PGSTAT_TYPE_DB, /* database-wide statistics */
+ PGSTAT_TYPE_TABLE, /* per-table statistics */
+ PGSTAT_TYPE_FUNCTION, /* per-function statistics */
+ PGSTAT_TYPE_REPLSLOT /* per-replication-slot statistics */
+} PgStatTypes;
/*
- * pgStatTabHash entry: map from relation OID to PgStat_TableStatus pointer
+ * entry body size lookup table of shared statistics entries corresponding to
+ * PgStatTypes
*/
-typedef struct TabStatHashEntry
+static const size_t pgstat_sharedentsize[] =
{
- Oid t_id;
- PgStat_TableStatus *tsa_entry;
-} TabStatHashEntry;
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_StatTabEntry), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_StatFuncEntry), /* PGSTAT_TYPE_FUNCTION */
+ sizeof(PgStat_ReplSlot) /* PGSTAT_TYPE_REPLSLOT */
+};
-/*
- * Hash table for O(1) t_id -> tsa_entry lookup
- */
-static HTAB *pgStatTabHash = NULL;
+/* Ditto for local statistics entries */
+static const size_t pgstat_localentsize[] =
+{
+ sizeof(PgStat_StatDBEntry), /* PGSTAT_TYPE_DB */
+ sizeof(PgStat_TableStatus), /* PGSTAT_TYPE_TABLE */
+ sizeof(PgStat_BackendFunctionEntry),/* PGSTAT_TYPE_FUNCTION */
+ sizeof(PgStat_ReplSlot) /* PGSTAT_TYPE_REPLSLOT */
+};
/*
- * Backends store per-function info that's waiting to be sent to the collector
- * in this hash table (indexed by function OID).
+ * We shoud avoid overwriting header part of a shared entry. Use these macros
+ * to know what portion of the struct to be written or read. PSTAT_SHENT_BODY
+ * returns a bit smaller address than the actual address of the next member but
+ * that doesn't matter.
*/
-static HTAB *pgStatFunctions = NULL;
+#define PGSTAT_SHENT_BODY(e) (((char *)(e)) + sizeof(PgStat_StatEntryHeader))
+#define PGSTAT_SHENT_BODY_LEN(t) \
+ (pgstat_sharedentsize[t] - sizeof(PgStat_StatEntryHeader))
+
+/* struct for shared statistics hash entry key. */
+typedef struct PgStatHashKey
+{
+ PgStatTypes type; /* statistics entry type */
+ Oid databaseid; /* database ID. InvalidOid for shared objects. */
+ Oid objectid; /* object ID, either table or function. */
+} PgStatHashKey;
+
+/* struct for shared statistics hash entry */
+typedef struct PgStatHashEntry
+{
+ PgStatHashKey key; /* hash key */
+ dsa_pointer body;/* pointer to shared stats in PgStat_StatEntryHeader */
+} PgStatHashEntry;
+
+/* struct for shared statistics local hash entry. */
+typedef struct PgStatLocalHashEntry
+{
+ PgStatHashKey key; /* hash key */
+ char status; /* for simplehash use */
+ PgStat_StatEntryHeader *body; /* pointer to stats body in local heap */
+ dsa_pointer dsapointer; /* dsa pointer of body */
+} PgStatLocalHashEntry;
+
+/* parameter for the shared hash */
+static const dshash_parameters dsh_params = {
+ sizeof(PgStatHashKey),
+ sizeof(PgStatHashEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_STATS
+};
+
+/* define hashtable for local hashes */
+#define SH_PREFIX pgstat_localhash
+#define SH_ELEMENT_TYPE PgStatLocalHashEntry
+#define SH_KEY_TYPE PgStatHashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ hash_bytes((unsigned char *)&key, sizeof(PgStatHashKey))
+#define SH_EQUAL(tb, a, b) (memcmp(&a, &b, sizeof(PgStatHashKey)) == 0)
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+/* The shared hash to index activity stats entries. */
+static dshash_table *pgStatSharedHash = NULL;
/*
- * Indicates if backend has some function stats that it hasn't yet
- * sent to the collector.
+ * The local cache to index shared stats entries.
+ *
+ * This is a local hash to store native pointers to shared hash
+ * entries. pgStatEntHashAge is copied from StatsShmem->gc_count at creation
+ * and garbage collection.
*/
-static bool have_function_stats = false;
+static pgstat_localhash_hash *pgStatEntHash = NULL;
+static int pgStatEntHashAge = 0; /* cache age of pgStatEntHash */
+
+/* Local stats numbers are stored here. */
+static pgstat_localhash_hash *pgStatLocalHash = NULL;
+
+/* entry type for oid hash */
+typedef struct pgstat_oident
+{
+ Oid oid;
+ char status;
+} pgstat_oident;
+
+/* Define hashtable for OID hashes. */
+StaticAssertDecl(sizeof(Oid) == 4, "oid is not compatible with uint32");
+#define SH_PREFIX pgstat_oid
+#define SH_ELEMENT_TYPE pgstat_oident
+#define SH_KEY_TYPE Oid
+#define SH_KEY oid
+#define SH_HASH_KEY(tb, key) hash_bytes_uint32(key)
+#define SH_EQUAL(tb, a, b) (a == b)
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
/*
* Tuple insertion/deletion counts for an open transaction can't be propagated
@@ -273,11 +411,8 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
-/*
- * Info about current "snapshot" of stats file
- */
+/* Variables for backend status snapshot */
static MemoryContext pgStatLocalContext = NULL;
-static HTAB *pgStatDBHash = NULL;
/* Status for backends including auxiliary */
static LocalPgBackendStatus *localBackendStatusTable = NULL;
@@ -286,23 +421,9 @@ static LocalPgBackendStatus *localBackendStatusTable = NULL;
static int localNumBackends = 0;
/*
- * Cluster wide statistics, kept in the stats collector.
- * Contains statistics that are not collected per database
- * or per table.
+ * Make our own memory context to make it easy to track memory usage.
*/
-static PgStat_ArchiverStats archiverStats;
-static PgStat_GlobalStats globalStats;
-static PgStat_WalStats walStats;
-static PgStat_SLRUStats slruStats[SLRU_NUM_ELEMENTS];
-static PgStat_ReplSlotStats *replSlotStats;
-static int nReplSlotStats;
-
-/*
- * List of OIDs of databases we need to write out. If an entry is InvalidOid,
- * it means to write only the shared-catalog stats ("DB 0"); otherwise, we
- * will write both that DB's data and the shared stats.
- */
-static List *pending_write_requests = NIL;
+MemoryContext pgStatCacheContext = NULL;
/*
* Total time charged to functions so far in the current backend.
@@ -311,40 +432,57 @@ static List *pending_write_requests = NIL;
*/
static instr_time total_func_time;
+/* Simple caching feature for pgstatfuncs */
+static PgStatHashKey stathashkey_zero = {0};
+static PgStatHashKey cached_dbent_key = {0};
+static PgStat_StatDBEntry cached_dbent;
+static PgStatHashKey cached_tabent_key = {0};
+static PgStat_StatTabEntry cached_tabent;
+static PgStatHashKey cached_funcent_key = {0};
+static PgStat_StatFuncEntry cached_funcent;
+
+static PgStat_Archiver cached_archiverstats;
+static bool cached_archiverstats_is_valid = false;
+static PgStat_BgWriter cached_bgwriterstats;
+static bool cached_bgwriterstats_is_valid = false;
+static PgStat_CheckPointer cached_checkpointerstats;
+static bool cached_checkpointerstats_is_valid = false;
+static PgStat_Wal cached_walstats;
+static bool cached_walstats_is_valid = false;
+static PgStat_SLRUStats cached_slrustats;
+static bool cached_slrustats_is_valid = false;
+static PgStat_ReplSlot *cached_replslotstats = NULL;
+static int n_cached_replslotstats = -1;
/* ----------
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
-static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
-static PgStat_StatTabEntry *pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry,
- Oid tableoid, bool create);
-static void pgstat_write_statsfiles(bool permanent, bool allDbs);
-static void pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent);
-static HTAB *pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep);
-static void pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash, bool permanent);
-static void backend_read_statsfile(void);
+static PgStat_StatDBEntry *get_local_dbstat_entry(Oid dbid);
+static PgStat_TableStatus *get_local_tabstat_entry(Oid rel_id, bool isshared);
+
+static void pgstat_write_statsfile(void);
+
+static void pgstat_read_statsfile(void);
static void pgstat_read_current_status(void);
-static bool pgstat_write_statsfile_needed(void);
-static bool pgstat_db_requested(Oid databaseid);
+static PgStat_StatEntryHeader * get_stat_entry(PgStatTypes type, Oid dbid,
+ Oid objid, bool nowait,
+ bool create, bool *found);
-static int pgstat_replslot_index(const char *name, bool create_it);
-static void pgstat_reset_replslot(int i, TimestampTz ts);
+static bool flush_tabstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_funcstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_dbstat(PgStatLocalHashEntry *ent, bool nowait);
+static bool flush_walstat(bool nowait);
+static bool flush_slrustat(bool nowait);
+static void delete_current_stats_entry(dshash_seq_status *hstat);
+static PgStat_StatEntryHeader * get_local_stat_entry(PgStatTypes type, Oid dbid,
+ Oid objid, bool create,
+ bool *found);
-static void pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg);
-static void pgstat_send_funcstats(void);
-static void pgstat_send_slru(void);
-static HTAB *pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid);
-
-static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
+static pgstat_oid_hash *collect_oids(Oid catalogid, AttrNumber anum_oid);
static void pgstat_setup_memcxt(void);
@@ -354,491 +492,645 @@ static const char *pgstat_get_wait_ipc(WaitEventIPC w);
static const char *pgstat_get_wait_timeout(WaitEventTimeout w);
static const char *pgstat_get_wait_io(WaitEventIO w);
-static void pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype);
-static void pgstat_send(void *msg, int len);
-
-static void pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len);
-static void pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len);
-static void pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len);
-static void pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len);
-static void pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len);
-static void pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len);
-static void pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len);
-static void pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len);
-static void pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg, int len);
-static void pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len);
-static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
-static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
-static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
-static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
-static void pgstat_recv_wal(PgStat_MsgWal *msg, int len);
-static void pgstat_recv_slru(PgStat_MsgSLRU *msg, int len);
-static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
-static void pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len);
-static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len);
-static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
-static void pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len);
-static void pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len);
-static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
-
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
*/
-/* ----------
- * pgstat_init() -
- *
- * Called from postmaster at startup. Create the resources required
- * by the statistics collector process. If unable to do so, do not
- * fail --- better to let the postmaster start with stats collection
- * disabled.
- * ----------
+/*
+ * StatsShmemSize
+ * Compute shared memory space needed for activity statistic
+ */
+Size
+StatsShmemSize(void)
+{
+ return sizeof(StatsShmemStruct);
+}
+
+/*
+ * StatsShmemInit - initialize during shared-memory creation
*/
void
-pgstat_init(void)
+StatsShmemInit(void)
{
- ACCEPT_TYPE_ARG3 alen;
- struct addrinfo *addrs = NULL,
- *addr,
- hints;
- int ret;
- fd_set rset;
- struct timeval tv;
- char test_byte;
- int sel_res;
- int tries = 0;
-
-#define TESTBYTEVAL ((char) 199)
+ bool found;
- /*
- * This static assertion verifies that we didn't mess up the calculations
- * involved in selecting maximum payload sizes for our UDP messages.
- * Because the only consequence of overrunning PGSTAT_MAX_MSG_SIZE would
- * be silent performance loss from fragmentation, it seems worth having a
- * compile-time cross-check that we didn't.
- */
- StaticAssertStmt(sizeof(PgStat_Msg) <= PGSTAT_MAX_MSG_SIZE,
- "maximum stats message size exceeds PGSTAT_MAX_MSG_SIZE");
+ StatsShmem = (StatsShmemStruct *)
+ ShmemInitStruct("Stats area", StatsShmemSize(), &found);
- /*
- * Create the UDP socket for sending and receiving statistic messages
- */
- hints.ai_flags = AI_PASSIVE;
- hints.ai_family = AF_UNSPEC;
- hints.ai_socktype = SOCK_DGRAM;
- hints.ai_protocol = 0;
- hints.ai_addrlen = 0;
- hints.ai_addr = NULL;
- hints.ai_canonname = NULL;
- hints.ai_next = NULL;
- ret = pg_getaddrinfo_all("localhost", NULL, &hints, &addrs);
- if (ret || !addrs)
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ ConditionVariableInit(&StatsShmem->holdoff_cv);
+ pg_atomic_init_u32(&StatsShmem->archiver_changecount, 0);
+ pg_atomic_init_u32(&StatsShmem->bgwriter_changecount, 0);
+ pg_atomic_init_u32(&StatsShmem->checkpointer_changecount, 0);
+
+ pg_atomic_init_u64(&StatsShmem->gc_count, 0);
+
+ LWLockInitialize(&StatsShmem->wal_stats_lock, LWTRANCHE_STATS);
+ }
+}
+
+/* ----------
+ * allow_next_attacher() -
+ *
+ * Let other processes to go ahead attaching the shared stats area.
+ * ----------
+ */
+static void
+allow_next_attacher(void)
+{
+ bool triggerd = false;
+
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (StatsShmem->attach_holdoff)
{
- ereport(LOG,
- (errmsg("could not resolve \"localhost\": %s",
- gai_strerror(ret))));
- goto startup_failed;
+ StatsShmem->attach_holdoff = false;
+ triggerd = true;
}
+ LWLockRelease(StatsLock);
+
+ if (triggerd)
+ ConditionVariableBroadcast(&StatsShmem->holdoff_cv);
+}
+
+/* ----------
+ * attach_shared_stats() -
+ *
+ * Attach shared or create stats memory. If we are the first process to use
+ * activity stats system, read the saved statistics file if any.
+ * ---------
+ */
+static void
+attach_shared_stats(void)
+{
+ MemoryContext oldcontext;
/*
- * On some platforms, pg_getaddrinfo_all() may return multiple addresses
- * only one of which will actually work (eg, both IPv6 and IPv4 addresses
- * when kernel will reject IPv6). Worse, the failure may occur at the
- * bind() or perhaps even connect() stage. So we must loop through the
- * results till we find a working combination. We will generate LOG
- * messages, but no error, for bogus combinations.
+ * Don't use dsm under postmaster, or when not tracking counts.
*/
- for (addr = addrs; addr; addr = addr->ai_next)
- {
-#ifdef HAVE_UNIX_SOCKETS
- /* Ignore AF_UNIX sockets, if any are returned. */
- if (addr->ai_family == AF_UNIX)
- continue;
-#endif
+ if (!pgstat_track_counts || !IsUnderPostmaster)
+ return;
- if (++tries > 1)
- ereport(LOG,
- (errmsg("trying another address for the statistics collector")));
+ pgstat_setup_memcxt();
- /*
- * Create the socket.
- */
- if ((pgStatSock = socket(addr->ai_family, SOCK_DGRAM, 0)) == PGINVALID_SOCKET)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not create socket for statistics collector: %m")));
- continue;
- }
+ if (area)
+ return;
- /*
- * Bind it to a kernel assigned port on localhost and get the assigned
- * port via getsockname().
- */
- if (bind(pgStatSock, addr->ai_addr, addr->ai_addrlen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not bind socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- alen = sizeof(pgStatAddr);
- if (getsockname(pgStatSock, (struct sockaddr *) &pgStatAddr, &alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not get address of socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /* stats shared memory persists for the backend lifetime */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- /*
- * Connect the socket to its own address. This saves a few cycles by
- * not having to respecify the target address on every send. This also
- * provides a kernel-level check that only packets from this same
- * address will be received.
- */
- if (connect(pgStatSock, (struct sockaddr *) &pgStatAddr, alen) < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not connect socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ /*
+ * The first attacher backend may still reading the stats file, or the last
+ * detacher may writing it. Wait for the work to finish.
+ */
+ ConditionVariablePrepareToSleep(&StatsShmem->holdoff_cv);
+ for (;;)
+ {
+ bool hold_off;
- /*
- * Try to send and receive a one-byte test message on the socket. This
- * is to catch situations where the socket can be created but will not
- * actually pass data (for instance, because kernel packet filtering
- * rules prevent it).
- */
- test_byte = TESTBYTEVAL;
-
-retry1:
- if (send(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not send test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
+ LWLockAcquire(StatsLock, LW_SHARED);
+ hold_off = StatsShmem->attach_holdoff;
+ LWLockRelease(StatsLock);
- /*
- * There could possibly be a little delay before the message can be
- * received. We arbitrarily allow up to half a second before deciding
- * it's broken.
- */
- for (;;) /* need a loop to handle EINTR */
- {
- FD_ZERO(&rset);
- FD_SET(pgStatSock, &rset);
-
- tv.tv_sec = 0;
- tv.tv_usec = 500000;
- sel_res = select(pgStatSock + 1, &rset, NULL, NULL, &tv);
- if (sel_res >= 0 || errno != EINTR)
- break;
- }
- if (sel_res < 0)
- {
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("select() failed in statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
- if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
- {
- /*
- * This is the case we actually think is likely, so take pains to
- * give a specific message for it.
- *
- * errno will not be set meaningfully here, so don't use it.
- */
- ereport(LOG,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("test message did not get through on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- test_byte++; /* just make sure variable is changed */
-
-retry2:
- if (recv(pgStatSock, &test_byte, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry2; /* if interrupted, just retry */
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not receive test message on socket for statistics collector: %m")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("incorrect test message transmission on socket for statistics collector")));
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
- continue;
- }
-
- /* If we get here, we have a working socket */
- break;
+ if (!hold_off)
+ break;
+
+ ConditionVariableTimedSleep(&StatsShmem->holdoff_cv, 10,
+ WAIT_EVENT_READING_STATS_FILE);
}
+ ConditionVariableCancelSleep();
- /* Did we find a working address? */
- if (!addr || pgStatSock == PGINVALID_SOCKET)
- goto startup_failed;
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
/*
- * Set the socket to non-blocking IO. This ensures that if the collector
- * falls behind, statistics messages will be discarded; backends won't
- * block waiting to send messages to the collector.
+ * The last process is responsible to write out stats files at exit.
+ * Maintain refcount so that a process going to exit can find whether it is
+ * the last one or not.
*/
- if (!pg_set_noblock(pgStatSock))
+ if (StatsShmem->refcount > 0)
+ StatsShmem->refcount++;
+ else
{
- ereport(LOG,
- (errcode_for_socket_access(),
- errmsg("could not set statistics collector socket to nonblocking mode: %m")));
- goto startup_failed;
+ /* We're the first process to attach the shared stats memory */
+ Assert(StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID);
+
+ /* Initialize shared memory area */
+ area = dsa_create(LWTRANCHE_STATS);
+ pgStatSharedHash = dshash_create(area, &dsh_params, 0);
+
+ StatsShmem->stats_dsa_handle = dsa_get_handle(area);
+ StatsShmem->hash_handle = dshash_get_hash_table_handle(pgStatSharedHash);
+ LWLockInitialize(&StatsShmem->slru_stats.lock, LWTRANCHE_STATS);
+ pg_atomic_init_u32(&StatsShmem->slru_stats.changecount, 0);
+
+ /* Block the next attacher for a while, see the comment above. */
+ StatsShmem->attach_holdoff = true;
+
+ StatsShmem->refcount = 1;
}
- /*
- * Try to ensure that the socket's receive buffer is at least
- * PGSTAT_MIN_RCVBUF bytes, so that it won't easily overflow and lose
- * data. Use of UDP protocol means that we are willing to lose data under
- * heavy load, but we don't want it to happen just because of ridiculously
- * small default buffer sizes (such as 8KB on older Windows versions).
- */
+ LWLockRelease(StatsLock);
+
+ if (area)
{
- int old_rcvbuf;
- int new_rcvbuf;
- ACCEPT_TYPE_ARG3 rcvbufsize = sizeof(old_rcvbuf);
-
- if (getsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &old_rcvbuf, &rcvbufsize) < 0)
- {
- ereport(LOG,
- (errmsg("getsockopt(%s) failed: %m", "SO_RCVBUF")));
- /* if we can't get existing size, always try to set it */
- old_rcvbuf = 0;
- }
-
- new_rcvbuf = PGSTAT_MIN_RCVBUF;
- if (old_rcvbuf < new_rcvbuf)
- {
- if (setsockopt(pgStatSock, SOL_SOCKET, SO_RCVBUF,
- (char *) &new_rcvbuf, sizeof(new_rcvbuf)) < 0)
- ereport(LOG,
- (errmsg("setsockopt(%s) failed: %m", "SO_RCVBUF")));
- }
+ /*
+ * We're the first attacher process, read stats file while blocking
+ * successors.
+ */
+ Assert(StatsShmem->attach_holdoff);
+ pgstat_read_statsfile();
+ allow_next_attacher();
+ }
+ else
+ {
+ /* We're not the first one, attach existing shared area. */
+ area = dsa_attach(StatsShmem->stats_dsa_handle);
+ pgStatSharedHash = dshash_attach(area, &dsh_params,
+ StatsShmem->hash_handle, 0);
}
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
- /* Now that we have a long-lived socket, tell fd.c about it. */
- ReserveExternalFD();
+ MemoryContextSwitchTo(oldcontext);
- return;
+ /* don't detach automatically */
+ dsa_pin_mapping(area);
+}
-startup_failed:
- ereport(LOG,
- (errmsg("disabling statistics collector for lack of working socket")));
+/* ----------
+ * cleanup_dropped_stats_entries() -
+ * Clean up shared stats entries no longer used.
+ *
+ * Shared stats entries for dropped objects may be left referenced. Clean up
+ * our reference and drop the shared entry if needed.
+ * ----------
+ */
+static void
+cleanup_dropped_stats_entries(void)
+{
+ pgstat_localhash_iterator i;
+ PgStatLocalHashEntry *ent;
- if (addrs)
- pg_freeaddrinfo_all(hints.ai_family, addrs);
+ if (pgStatEntHash == NULL)
+ return;
- if (pgStatSock != PGINVALID_SOCKET)
- closesocket(pgStatSock);
- pgStatSock = PGINVALID_SOCKET;
+ pgstat_localhash_start_iterate(pgStatEntHash, &i);
+ while ((ent = pgstat_localhash_iterate(pgStatEntHash, &i))
+ != NULL)
+ {
+ /*
+ * Free the shared memory chunk for the entry if we were the last
+ * referrer to a dropped entry.
+ */
+ if (pg_atomic_sub_fetch_u32(&ent->body->refcount, 1) < 1 &&
+ ent->body->dropped)
+ dsa_free(area, ent->dsapointer);
+ }
/*
- * Adjust GUC variables to suppress useless activity, and for debugging
- * purposes (seeing track_counts off is a clue that we failed here). We
- * use PGC_S_OVERRIDE because there is no point in trying to turn it back
- * on from postgresql.conf without a restart.
+ * This function is expected to be called during backend exit. So we don't
+ * bother destroying pgStatEntHash.
*/
- SetConfigOption("track_counts", "off", PGC_INTERNAL, PGC_S_OVERRIDE);
+ pgStatEntHash = NULL;
}
-/*
- * subroutine for pgstat_reset_all
+/* ----------
+ * detach_shared_stats() -
+ *
+ * Detach shared stats. Write out to file if we're the last process and told
+ * to do so.
+ * ----------
*/
static void
-pgstat_reset_remove_files(const char *directory)
+detach_shared_stats(bool write_file)
{
- DIR *dir;
- struct dirent *entry;
- char fname[MAXPGPATH * 2];
+ bool is_last_detacher = 0;
+
+ /* immediately return if useless */
+ if (!area || !IsUnderPostmaster)
+ return;
+
+ /* We shouldn't leave a reference to shared stats. */
+ cleanup_dropped_stats_entries();
- dir = AllocateDir(directory);
- while ((entry = ReadDir(dir, directory)) != NULL)
+ /*
+ * If we are the last detacher, hold off the next attacher (if possible)
+ * until we finish writing stats file.
+ */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+ if (--StatsShmem->refcount == 0)
{
- int nchars;
- Oid tmp_oid;
+ StatsShmem->attach_holdoff = true;
+ is_last_detacher = true;
+ }
+ LWLockRelease(StatsLock);
- /*
- * Skip directory entries that don't match the file names we write.
- * See get_dbstat_filename for the database-specific pattern.
- */
- if (strncmp(entry->d_name, "global.", 7) == 0)
- nchars = 7;
- else
- {
- nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
- /* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
- continue;
- }
-
- if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
- strcmp(entry->d_name + nchars, "stat") != 0)
- continue;
-
- snprintf(fname, sizeof(fname), "%s/%s", directory,
- entry->d_name);
- unlink(fname);
+ if (is_last_detacher)
+ {
+ if (write_file)
+ pgstat_write_statsfile();
+
+ StatsShmem->stats_dsa_handle = DSM_HANDLE_INVALID;
+ /* allow the next attacher, if any */
+ allow_next_attacher();
}
- FreeDir(dir);
+
+ /*
+ * Detach the area. It is automatically destroyed when the last process
+ * detached it.
+ */
+ dsa_detach(area);
+
+ area = NULL;
+ pgStatSharedHash = NULL;
+
+ /* We are going to exit. Don't bother destroying local hashes. */
+ pgStatLocalHash = NULL;
}
/*
* pgstat_reset_all() -
*
- * Remove the stats files. This is currently used only if WAL
- * recovery is needed after a crash.
+ * Remove the stats file. This is currently used only if WAL recovery is
+ * needed after a crash.
*/
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
+ /* standalone server doesn't use shared stats */
+ if (!IsUnderPostmaster)
+ return;
+
+ /* we must have shared stats attached */
+ Assert(StatsShmem->stats_dsa_handle != DSM_HANDLE_INVALID);
+
+ /* Startup must be the only user of shared stats */
+ Assert(StatsShmem->refcount == 1);
+
+ /*
+ * We could directly remove files and recreate the shared memory area. But
+ * just discard then create for simplicity.
+ */
+ detach_shared_stats(false);
+ attach_shared_stats();
}
-#ifdef EXEC_BACKEND
/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
+ * fetch_lock_statentry - common helper function to fetch and lock a stats
+ * entry for flush_tabstat, flush_funcstat and flush_dbstat.
*/
-static pid_t
-pgstat_forkexec(void)
+static PgStat_StatEntryHeader *
+fetch_lock_statentry(PgStatTypes type, Oid dboid, Oid objid, bool nowait)
{
- char *av[10];
- int ac = 0;
+ PgStat_StatEntryHeader *header;
+
+ /* find shared table stats entry corresponding to the local entry */
+ header = (PgStat_StatEntryHeader *)
+ get_stat_entry(type, dboid, objid, nowait, true, NULL);
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ /* skip if dshash failed to acquire lock */
+ if (header == NULL)
+ return false;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ return false;
- return postmaster_forkexec(ac, av);
+ return header;
}
-#endif /* EXEC_BACKEND */
-/*
- * pgstat_start() -
+/* ----------
+ * get_stat_entry() -
*
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
+ * get shared stats entry for specified type, dbid and objid.
+ * If nowait is true, returns NULL on lock failure.
+ *
+ * If initfunc is not NULL, new entry is created if not yet and the function
+ * is called with the new base entry. If found is not NULL, it is set to true
+ * if existing entry is found or false if not.
+ * ----------
+ */
+static PgStat_StatEntryHeader *
+get_stat_entry(PgStatTypes type, Oid dbid, Oid objid, bool nowait, bool create,
+ bool *found)
+{
+ PgStatHashEntry *shhashent;
+ PgStatLocalHashEntry *lohashent;
+ PgStat_StatEntryHeader *shheader = NULL;
+ PgStatHashKey key;
+ bool shfound;
+
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+
+ if (pgStatEntHash)
+ {
+ uint64 currage;
+
+ /*
+ * pgStatEntHashAge increments quite slowly than the time the following
+ * loop takes so this is expected to iterate no more than twice.
+ */
+ while (unlikely
+ (pgStatEntHashAge !=
+ (currage = pg_atomic_read_u64(&StatsShmem->gc_count))))
+ {
+ pgstat_localhash_iterator i;
+
+ /*
+ * Some entries have been dropped. Invalidate cache pointer to
+ * them.
+ */
+ pgstat_localhash_start_iterate(pgStatEntHash, &i);
+ while ((lohashent = pgstat_localhash_iterate(pgStatEntHash, &i))
+ != NULL)
+ {
+ PgStat_StatEntryHeader *header = lohashent->body;
+ if (header->dropped)
+ {
+ pgstat_localhash_delete(pgStatEntHash, key);
+
+ if (pg_atomic_sub_fetch_u32(&header->refcount, 1) < 1)
+ {
+ /*
+ * We're the last referrer to this entry, drop the
+ * shared entry.
+ */
+ dsa_free(area, lohashent->dsapointer);
+ }
+ }
+ }
+
+ pgStatEntHashAge = currage;
+ }
+
+ lohashent = pgstat_localhash_lookup(pgStatEntHash, key);
+
+ if (lohashent)
+ {
+ if (found)
+ *found = true;
+ return lohashent->body;
+ }
+ }
+
+ shhashent = dshash_find_extended(pgStatSharedHash, &key,
+ create, nowait, create, &shfound);
+ if (shhashent)
+ {
+ if (create && !shfound)
+ {
+ /* Create new stats entry. */
+ dsa_pointer chunk = dsa_allocate0(area,
+ pgstat_sharedentsize[type]);
+
+ shheader = dsa_get_address(area, chunk);
+ LWLockInitialize(&shheader->lock, LWTRANCHE_STATS);
+ pg_atomic_init_u32(&shheader->refcount, 0);
+
+ /* Link the new entry from the hash entry. */
+ shhashent->body = chunk;
+ }
+ else
+ shheader = dsa_get_address(area, shhashent->body);
+
+ /*
+ * We expose this shared entry now. You might think that the entry can
+ * be removed by a concurrent backend, but since we are creating an
+ * stats entry, the object actually exists and used in the upper
+ * layer. Such an object cannot be dropped until the first vacuum after
+ * the current transaction ends.
+ */
+ dshash_release_lock(pgStatSharedHash, shhashent);
+
+ /* register to local hash if possible */
+ if (pgStatEntHash || pgStatCacheContext)
+ {
+ bool lofound;
+
+ if (pgStatEntHash == NULL)
+ {
+ pgStatEntHash =
+ pgstat_localhash_create(pgStatCacheContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
+ pgStatEntHashAge =
+ pg_atomic_read_u64(&StatsShmem->gc_count);
+ }
+
+ lohashent =
+ pgstat_localhash_insert(pgStatEntHash, key, &lofound);
+
+ Assert(!lofound);
+ lohashent->body = shheader;
+ lohashent->dsapointer = shhashent->body;
+
+ pg_atomic_add_fetch_u32(&shheader->refcount, 1);
+ }
+ }
+
+ if (found)
+ *found = shfound;
+
+ return shheader;
+}
+
+/*
+ * flush_walstat - flush out a local WAL stats entries
*
- * Returns PID of child process, or 0 if fail.
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
*
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Returns true if all local WAL stats are successfully flushed out.
*/
-int
-pgstat_start(void)
+static bool
+flush_walstat(bool nowait)
{
- time_t curtime;
- pid_t pgStatPid;
+ PgStat_Wal *s = &StatsShmem->wal_stats;
+ PgStat_Wal *l = &WalStats;
+ WalUsage all_zeroes = {0} PG_USED_FOR_ASSERTS_ONLY;
+
+ /*
+ * We don't update the WAL usage portion of the local WalStats
+ * elsewhere. Instead, fill in that portion with the difference of
+ * pgWalUsage since the previous call.
+ */
+ Assert(memcmp(&l->wal_usage, &all_zeroes, sizeof(WalUsage)) == 0);
+ WalUsageAccumDiff(&l->wal_usage, &pgWalUsage, &prevWalUsage);
/*
- * Check that the socket is there, else pgstat_init failed and we can do
- * nothing useful.
+ * This function can be called even if nothing at all has happened. Avoid
+ * taking lock for nothing in that case.
*/
- if (pgStatSock == PGINVALID_SOCKET)
- return 0;
+ if (!walstats_pending())
+ return true;
+
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&StatsShmem->wal_stats_lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&StatsShmem->wal_stats_lock,
+ LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
+
+ s->wal_usage.wal_records += l->wal_usage.wal_records;
+ s->wal_usage.wal_fpi += l->wal_usage.wal_fpi;
+ s->wal_usage.wal_bytes += l->wal_usage.wal_bytes;
+ s->wal_buffers_full += l->wal_buffers_full;
+ LWLockRelease(&StatsShmem->wal_stats_lock);
/*
- * Do nothing if too soon since last collector start. This is a safety
- * valve to protect against continuous respawn attempts if the collector
- * is dying immediately at launch. Note that since we will be re-called
- * from the postmaster main loop, we will get another chance later.
+ * Save the current counters for the subsequent calculation of WAL usage.
*/
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
- (unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ prevWalUsage = pgWalUsage;
/*
- * Okay, fork off the collector.
+ * Clear out the statistics buffer, so it can be re-used.
*/
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
+ MemSet(&WalStats, 0, sizeof(WalStats));
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
+ return true;
+}
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+/*
+ * flush_slrustat - flush out a local SLRU stats entries
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true. Writer processes are mutually excluded
+ * using LWLock, but readers are expected to use change-count protocol to avoid
+ * interference with writers.
+ *
+ * Returns true if all local SLRU stats are successfully flushed out.
+ */
+static bool
+flush_slrustat(bool nowait)
+{
+ uint32 assert_changecount PG_USED_FOR_ASSERTS_ONLY;
+ int i;
+
+ if (!have_slrustats)
+ return true;
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ /* lock the shared entry to protect the content, skip if failed */
+ if (!nowait)
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ else if (!LWLockConditionalAcquire(&StatsShmem->slru_stats.lock,
+ LW_EXCLUSIVE))
+ return false; /* failed to acquire lock, skip */
- PgstatCollectorMain(0, NULL);
- break;
-#endif
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_stats.changecount, 1);
+ Assert((assert_changecount & 1) == 0);
- default:
- return (int) pgStatPid;
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS ; i++)
+ {
+ PgStat_SLRUStats *sharedent = &StatsShmem->slru_stats.entry[i];
+ PgStat_SLRUStats *localent = &local_SLRUStats[i];
+
+ sharedent->blocks_zeroed += localent->blocks_zeroed;
+ sharedent->blocks_hit += localent->blocks_hit;
+ sharedent->blocks_read += localent->blocks_read;
+ sharedent->blocks_written += localent->blocks_written;
+ sharedent->blocks_exists += localent->blocks_exists;
+ sharedent->flush += localent->flush;
+ sharedent->truncate += localent->truncate;
}
- /* shouldn't get here */
- return 0;
+ /* done, clear the local entry */
+ MemSet(local_SLRUStats, 0,
+ sizeof(PgStat_SLRUStats) * SLRU_NUM_ELEMENTS);
+
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_stats.changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+
+ have_slrustats = false;
+
+ return true;
+}
+
+/* ----------
+ * delete_current_stats_entry()
+ *
+ * Deletes the given shared entry from shared stats hash. The entry must be
+ * exclusively locked.
+ * ----------
+ */
+static void
+delete_current_stats_entry(dshash_seq_status *hstat)
+{
+ dsa_pointer pdsa;
+ PgStat_StatEntryHeader *header;
+ PgStatHashEntry *ent;
+
+ ent = dshash_get_current(hstat);
+ pdsa = ent->body;
+ header = dsa_get_address(area, pdsa);
+
+ /* No one find this entry ever after. */
+ dshash_delete_current(hstat);
+
+ /*
+ * Let the referrers drop the entry if any. Refcount won't be decremented
+ * until "dropped" is set true and StatsShmem->gc_count is incremented
+ * later. So we can check refcount to set dropped without holding a
+ * lock. If no one is referring this entry, free it immediately.
+ */
+
+ if (pg_atomic_read_u32(&header->refcount) > 0)
+ header->dropped = true;
+ else
+ dsa_free(area, pdsa);
+
+ return;
}
-void
-allow_immediate_pgstat_restart(void)
+/* ----------
+ * get_local_stat_entry() -
+ *
+ * Returns local stats entry for the type, dbid and objid.
+ * If create is true, new entry is created if not yet. found must be non-null
+ * in the case.
+ *
+ *
+ * The caller is responsible to initialize the statsbody part of the returned
+ * memory.
+ * ----------
+ */
+static PgStat_StatEntryHeader *
+get_local_stat_entry(PgStatTypes type, Oid dbid, Oid objid,
+ bool create, bool *found)
{
- last_pgstat_start_time = 0;
+ PgStatHashKey key;
+ PgStatLocalHashEntry *entry;
+
+ if (pgStatLocalHash == NULL)
+ pgStatLocalHash = pgstat_localhash_create(pgStatCacheContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
+
+ /* Find an entry or create a new one. */
+ key.type = type;
+ key.databaseid = dbid;
+ key.objectid = objid;
+ if (create)
+ entry = pgstat_localhash_insert(pgStatLocalHash, key, found);
+ else
+ entry = pgstat_localhash_lookup(pgStatLocalHash, key);
+
+ if (!create && !entry)
+ return NULL;
+
+ if (create && !*found)
+ entry->body = MemoryContextAllocZero(TopMemoryContext,
+ pgstat_localentsize[type]);
+
+ return entry->body;
}
/* ------------------------------------------------------------
@@ -846,150 +1138,399 @@ allow_immediate_pgstat_restart(void)
*------------------------------------------------------------
*/
-
/* ----------
* pgstat_report_stat() -
*
* Must be called by processes that performs DML: tcop/postgres.c, logical
- * receiver processes, SPI worker, etc. to send the so far collected
- * per-table and function usage statistics to the collector. Note that this
- * is called only when not within a transaction, so it is fair to use
+ * receiver processes, SPI worker, etc. to apply the so far collected
+ * per-table and function usage statistics to the shared statistics hashes.
+ *
+ * Updates are applied not more frequent than the interval of
+ * PGSTAT_MIN_INTERVAL milliseconds. They are also postponed on lock
+ * failure if force is false and there's no pending updates longer than
+ * PGSTAT_MAX_INTERVAL milliseconds. Postponed updates are retried in
+ * succeeding calls of this function.
+ *
+ * Returns the time until the next timing when updates are applied in
+ * milliseconds if there are no updates held for more than
+ * PGSTAT_MIN_INTERVAL milliseconds.
+ *
+ * Note that this is called only out of a transaction, so it is fine to use
* transaction stop time as an approximation of current time.
- * ----------
+ * ----------
*/
-void
+long
pgstat_report_stat(bool force)
{
- /* we assume this inits to all zeroes: */
- static const PgStat_TableCounts all_zeroes;
- static TimestampTz last_report = 0;
-
+ static TimestampTz next_flush = 0;
+ static TimestampTz pending_since = 0;
+ static long retry_interval = 0;
TimestampTz now;
- PgStat_MsgTabstat regular_msg;
- PgStat_MsgTabstat shared_msg;
- TabStatusArray *tsa;
+ bool nowait;
int i;
+ uint64 oldval;
+
+ /* Return if not active */
+ if (area == NULL)
+ return 0;
+
+ /*
+ * We need a database entry if the following stats exists.
+ */
+ if (pgStatXactCommit > 0 || pgStatXactRollback > 0 ||
+ pgStatBlockReadTime > 0 || pgStatBlockWriteTime > 0)
+ get_local_dbstat_entry(MyDatabaseId);
/* Don't expend a clock check if nothing to do */
- if ((pgStatTabList == NULL || pgStatTabList->tsa_used == 0) &&
- pgStatXactCommit == 0 && pgStatXactRollback == 0 &&
- !have_function_stats)
- return;
+ if (pgStatLocalHash == NULL && have_slrustats && !walstats_pending())
+ return 0;
- /*
- * Don't send a message unless it's been at least PGSTAT_STAT_INTERVAL
- * msec since we last sent one, or the caller wants to force stats out.
- */
now = GetCurrentTransactionStopTimestamp();
- if (!force &&
- !TimestampDifferenceExceeds(last_report, now, PGSTAT_STAT_INTERVAL))
- return;
- last_report = now;
-
- /*
- * Destroy pgStatTabHash before we start invalidating PgStat_TableEntry
- * entries it points to. (Should we fail partway through the loop below,
- * it's okay to have removed the hashtable already --- the only
- * consequence is we'd get multiple entries for the same table in the
- * pgStatTabList, and that's safe.)
- */
- if (pgStatTabHash)
- hash_destroy(pgStatTabHash);
- pgStatTabHash = NULL;
-
- /*
- * Scan through the TabStatusArray struct(s) to find tables that actually
- * have counts, and build messages to send. We have to separate shared
- * relations from regular ones because the databaseid field in the message
- * header has to depend on that.
- */
- regular_msg.m_databaseid = MyDatabaseId;
- shared_msg.m_databaseid = InvalidOid;
- regular_msg.m_nentries = 0;
- shared_msg.m_nentries = 0;
-
- for (tsa = pgStatTabList; tsa != NULL; tsa = tsa->tsa_next)
+
+ if (!force)
{
- for (i = 0; i < tsa->tsa_used; i++)
+ /*
+ * Don't flush stats too frequently. Return the time to the next
+ * flush.
+ */
+ if (now < next_flush)
{
- PgStat_TableStatus *entry = &tsa->tsa_entries[i];
- PgStat_MsgTabstat *this_msg;
- PgStat_TableEntry *this_ent;
-
- /* Shouldn't have any pending transaction-dependent counts */
- Assert(entry->trans == NULL);
-
- /*
- * Ignore entries that didn't accumulate any actual counts, such
- * as indexes that were opened by the planner but not used.
- */
- if (memcmp(&entry->t_counts, &all_zeroes,
- sizeof(PgStat_TableCounts)) == 0)
+ /* Record the epoch time if retrying. */
+ if (pending_since == 0)
+ pending_since = now;
+
+ return (next_flush - now) / 1000;
+ }
+
+ /* But, don't keep pending updates longer than PGSTAT_MAX_INTERVAL. */
+
+ if (pending_since > 0 &&
+ TimestampDifferenceExceeds(pending_since, now, PGSTAT_MAX_INTERVAL))
+ force = true;
+ }
+
+ /* don't wait for lock acquisition when !force */
+ nowait = !force;
+
+ if (pgStatLocalHash)
+ {
+ int remains = 0;
+ pgstat_localhash_iterator i;
+ List *dbentlist = NIL;
+ ListCell *lc;
+ PgStatLocalHashEntry *lent;
+
+ /* Step 1: flush out other than database stats */
+ pgstat_localhash_start_iterate(pgStatLocalHash, &i);
+ while ((lent = pgstat_localhash_iterate(pgStatLocalHash, &i)) != NULL)
+ {
+ bool remove = false;
+
+ switch (lent->key.type)
+ {
+ case PGSTAT_TYPE_DB:
+ /*
+ * flush_tabstat applies some of stats numbers of flushed
+ * entries into local database stats. Just remember the
+ * database entries for now then flush-out them later.
+ */
+ dbentlist = lappend(dbentlist, lent);
+ break;
+ case PGSTAT_TYPE_TABLE:
+ if (flush_tabstat(lent, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_FUNCTION:
+ if (flush_funcstat(lent, nowait))
+ remove = true;
+ break;
+ case PGSTAT_TYPE_REPLSLOT:
+ /* We don't have that kind of local entry */
+ Assert(false);
+ }
+
+ if (!remove)
+ {
+ remains++;
continue;
+ }
- /*
- * OK, insert data into the appropriate message, and send if full.
- */
- this_msg = entry->t_shared ? &shared_msg : ®ular_msg;
- this_ent = &this_msg->m_entry[this_msg->m_nentries];
- this_ent->t_id = entry->t_id;
- memcpy(&this_ent->t_counts, &entry->t_counts,
- sizeof(PgStat_TableCounts));
- if (++this_msg->m_nentries >= PGSTAT_NUM_TABENTRIES)
+ /* Remove the successfully flushed entry */
+ pfree(lent->body);
+ lent->body = NULL;
+ pgstat_localhash_delete(pgStatLocalHash, lent->key);
+ }
+
+ /* Step 2: flush out database stats */
+ foreach(lc, dbentlist)
+ {
+ PgStatLocalHashEntry *lent = (PgStatLocalHashEntry *) lfirst(lc);
+
+ if (flush_dbstat(lent, nowait))
{
- pgstat_send_tabstat(this_msg);
- this_msg->m_nentries = 0;
+ remains--;
+ /* Remove the successfully flushed entry */
+ pfree(lent->body);
+ lent->body = NULL;
+ pgstat_localhash_delete(pgStatLocalHash, lent->key);
}
}
- /* zero out PgStat_TableStatus structs after use */
- MemSet(tsa->tsa_entries, 0,
- tsa->tsa_used * sizeof(PgStat_TableStatus));
- tsa->tsa_used = 0;
+ list_free(dbentlist);
+ dbentlist = NULL;
+
+ if (remains <= 0)
+ {
+ pgstat_localhash_destroy(pgStatLocalHash);
+ pgStatLocalHash = NULL;
+ }
+ }
+
+ /* flush wal stats */
+ flush_walstat(nowait);
+
+ /* flush SLRU stats */
+ flush_slrustat(nowait);
+
+ /*
+ * Publish the time of the last flush, but we don't notify the change of
+ * the timestamp itself. Readers will get sufficiently recent timestamp.
+ * If we failed to update the value, concurrent processes should have
+ * updated it to sufficiently recent time.
+ *
+ * XXX: The loop might be unnecessary for the reason above.
+ */
+ oldval = pg_atomic_read_u64(&StatsShmem->stats_timestamp);
+
+ for (i = 0 ; i < 10 ; i++)
+ {
+ if (oldval >= now ||
+ pg_atomic_compare_exchange_u64(&StatsShmem->stats_timestamp,
+ &oldval, (uint64) now))
+ break;
}
/*
- * Send partial messages. Make sure that any pending xact commit/abort
- * gets counted, even if there are no table stats to send.
+ * Some of the local stats may have not been flushed due to lock
+ * contention. If we have such pending local stats here, let the caller
+ * know the retry interval.
*/
- if (regular_msg.m_nentries > 0 ||
- pgStatXactCommit > 0 || pgStatXactRollback > 0)
- pgstat_send_tabstat(®ular_msg);
- if (shared_msg.m_nentries > 0)
- pgstat_send_tabstat(&shared_msg);
+ if (pgStatLocalHash != NULL || have_slrustats || walstats_pending())
+ {
+ /* Retain the epoch time */
+ if (pending_since == 0)
+ pending_since = now;
- /* Now, send function statistics */
- pgstat_send_funcstats();
+ /* The interval is doubled at every retry. */
+ if (retry_interval == 0)
+ retry_interval = PGSTAT_RETRY_MIN_INTERVAL * 1000;
+ else
+ retry_interval = retry_interval * 2;
- /* Send WAL statistics */
- pgstat_send_wal();
+ /*
+ * Determine the next retry interval so as not to get shorter than the
+ * previous interval.
+ */
+ if (!TimestampDifferenceExceeds(pending_since,
+ now + 2 * retry_interval,
+ PGSTAT_MAX_INTERVAL))
+ next_flush = now + retry_interval;
+ else
+ {
+ next_flush = pending_since + PGSTAT_MAX_INTERVAL * 1000;
+ retry_interval = next_flush - now;
+ }
- /* Finally send SLRU statistics */
- pgstat_send_slru();
+ return retry_interval / 1000;
+ }
+
+ /* Set the next time to update stats */
+ next_flush = now + PGSTAT_MIN_INTERVAL * 1000;
+ retry_interval = 0;
+ pending_since = 0;
+
+ return 0;
}
/*
- * Subroutine for pgstat_report_stat: finish and send a tabstat message
+ * flush_tabstat - flush out a local table stats entry
+ *
+ * Some of the stats numbers are copied to local database stats entry after
+ * successful flush-out.
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
*/
-static void
-pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
+static bool
+flush_tabstat(PgStatLocalHashEntry *ent, bool nowait)
{
- int n;
- int len;
+ static const PgStat_TableCounts all_zeroes;
+ Oid dboid; /* database OID of the table */
+ PgStat_TableStatus *lstats; /* local stats entry */
+ PgStat_StatTabEntry *shtabstats; /* table entry of shared stats */
+ PgStat_StatDBEntry *ldbstats; /* local database entry */
+
+ Assert(ent->key.type == PGSTAT_TYPE_TABLE);
+ lstats = (PgStat_TableStatus *) ent->body;
+ dboid = ent->key.databaseid;
+
+ /*
+ * Ignore entries that didn't accumulate any actual counts, such as
+ * indexes that were opened by the planner but not used.
+ */
+ if (memcmp(&lstats->t_counts, &all_zeroes,
+ sizeof(PgStat_TableCounts)) == 0)
+ {
+ /* This local entry is going to be dropped, delink from relcache. */
+ pgstat_delinkstats(lstats->relation);
+ return true;
+ }
+
+ /* find shared table stats entry corresponding to the local entry */
+ shtabstats = (PgStat_StatTabEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_TABLE, dboid, ent->key.objectid,
+ nowait);
+
+ if (shtabstats == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ /* add the values to the shared entry. */
+ shtabstats->numscans += lstats->t_counts.t_numscans;
+ shtabstats->tuples_returned += lstats->t_counts.t_tuples_returned;
+ shtabstats->tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ shtabstats->tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ shtabstats->tuples_updated += lstats->t_counts.t_tuples_updated;
+ shtabstats->tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ shtabstats->tuples_hot_updated += lstats->t_counts.t_tuples_hot_updated;
+
+ /*
+ * If table was truncated or vacuum/analyze has ran, first reset the
+ * live/dead counters.
+ */
+ if (lstats->t_counts.t_truncated)
+ {
+ shtabstats->n_live_tuples = 0;
+ shtabstats->n_dead_tuples = 0;
+ }
+
+ shtabstats->n_live_tuples += lstats->t_counts.t_delta_live_tuples;
+ shtabstats->n_dead_tuples += lstats->t_counts.t_delta_dead_tuples;
+ shtabstats->changes_since_analyze += lstats->t_counts.t_changed_tuples;
+ shtabstats->inserts_since_vacuum += lstats->t_counts.t_tuples_inserted;
+ shtabstats->blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ shtabstats->blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /* Clamp n_live_tuples in case of negative delta_live_tuples */
+ shtabstats->n_live_tuples = Max(shtabstats->n_live_tuples, 0);
+ /* Likewise for n_dead_tuples */
+ shtabstats->n_dead_tuples = Max(shtabstats->n_dead_tuples, 0);
+
+ LWLockRelease(&shtabstats->header.lock);
+
+ /* The entry is successfully flushed so the same to add to database stats */
+ ldbstats = get_local_dbstat_entry(dboid);
+ ldbstats->counts.n_tuples_returned += lstats->t_counts.t_tuples_returned;
+ ldbstats->counts.n_tuples_fetched += lstats->t_counts.t_tuples_fetched;
+ ldbstats->counts.n_tuples_inserted += lstats->t_counts.t_tuples_inserted;
+ ldbstats->counts.n_tuples_updated += lstats->t_counts.t_tuples_updated;
+ ldbstats->counts.n_tuples_deleted += lstats->t_counts.t_tuples_deleted;
+ ldbstats->counts.n_blocks_fetched += lstats->t_counts.t_blocks_fetched;
+ ldbstats->counts.n_blocks_hit += lstats->t_counts.t_blocks_hit;
+
+ /* This local entry is going to be dropped, delink from relcache. */
+ pgstat_delinkstats(lstats->relation);
+
+ return true;
+}
+
+/*
+ * flush_funcstat - flush out a local function stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_funcstat(PgStatLocalHashEntry *ent, bool nowait)
+{
+ PgStat_BackendFunctionEntry *localent; /* local stats entry */
+ PgStat_StatFuncEntry *shfuncent = NULL; /* shared stats entry */
+
+ Assert(ent->key.type == PGSTAT_TYPE_FUNCTION);
+ localent = (PgStat_BackendFunctionEntry *) ent->body;
+
+ /* localent always has non-zero content */
+
+ /* find shared table stats entry corresponding to the local entry */
+ shfuncent = (PgStat_StatFuncEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ ent->key.objectid, nowait);
+ if (shfuncent == NULL)
+ return false; /* failed to acquire lock, skip */
+
+ shfuncent->f_numcalls += localent->f_counts.f_numcalls;
+ shfuncent->f_total_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+ shfuncent->f_self_time +=
+ INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+
+ LWLockRelease(&shfuncent->header.lock);
+
+ return true;
+}
+
+
+/*
+ * flush_dbstat - flush out a local database stats entry
+ *
+ * If nowait is true, this function returns false on lock failure. Otherwise
+ * this function always returns true.
+ *
+ * Returns true if the entry is successfully flushed out.
+ */
+static bool
+flush_dbstat(PgStatLocalHashEntry *ent, bool nowait)
+{
+ PgStat_StatDBEntry *localent;
+ PgStat_StatDBEntry *sharedent;
+
+ Assert(ent->key.type == PGSTAT_TYPE_DB);
+
+ localent = (PgStat_StatDBEntry *) &ent->body;
+
+ /* find shared database stats entry corresponding to the local entry */
+ sharedent = (PgStat_StatDBEntry *)
+ fetch_lock_statentry(PGSTAT_TYPE_DB, ent->key.databaseid, InvalidOid,
+ nowait);
+
+ if (!sharedent)
+ return false; /* failed to acquire lock, skip */
+
+ sharedent->counts.n_tuples_returned += localent->counts.n_tuples_returned;
+ sharedent->counts.n_tuples_fetched += localent->counts.n_tuples_fetched;
+ sharedent->counts.n_tuples_inserted += localent->counts.n_tuples_inserted;
+ sharedent->counts.n_tuples_updated += localent->counts.n_tuples_updated;
+ sharedent->counts.n_tuples_deleted += localent->counts.n_tuples_deleted;
+ sharedent->counts.n_blocks_fetched += localent->counts.n_blocks_fetched;
+ sharedent->counts.n_blocks_hit += localent->counts.n_blocks_hit;
- /* It's unlikely we'd get here with no socket, but maybe not impossible */
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ sharedent->counts.n_deadlocks += localent->counts.n_deadlocks;
+ sharedent->counts.n_temp_bytes += localent->counts.n_temp_bytes;
+ sharedent->counts.n_temp_files += localent->counts.n_temp_files;
+ sharedent->counts.n_checksum_failures += localent->counts.n_checksum_failures;
/*
- * Report and reset accumulated xact commit/rollback and I/O timings
- * whenever we send a normal tabstat message
+ * Accumulate xact commit/rollback and I/O timings to stats entry of the
+ * current database.
*/
- if (OidIsValid(tsmsg->m_databaseid))
+ if (OidIsValid(ent->key.databaseid))
{
- tsmsg->m_xact_commit = pgStatXactCommit;
- tsmsg->m_xact_rollback = pgStatXactRollback;
- tsmsg->m_block_read_time = pgStatBlockReadTime;
- tsmsg->m_block_write_time = pgStatBlockWriteTime;
+ sharedent->counts.n_xact_commit += pgStatXactCommit;
+ sharedent->counts.n_xact_rollback += pgStatXactRollback;
+ sharedent->counts.n_block_read_time += pgStatBlockReadTime;
+ sharedent->counts.n_block_write_time += pgStatBlockWriteTime;
pgStatXactCommit = 0;
pgStatXactRollback = 0;
pgStatBlockReadTime = 0;
@@ -997,281 +1538,138 @@ pgstat_send_tabstat(PgStat_MsgTabstat *tsmsg)
}
else
{
- tsmsg->m_xact_commit = 0;
- tsmsg->m_xact_rollback = 0;
- tsmsg->m_block_read_time = 0;
- tsmsg->m_block_write_time = 0;
+ sharedent->counts.n_xact_commit = 0;
+ sharedent->counts.n_xact_rollback = 0;
+ sharedent->counts.n_block_read_time = 0;
+ sharedent->counts.n_block_write_time = 0;
}
- n = tsmsg->m_nentries;
- len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
- n * sizeof(PgStat_TableEntry);
+ LWLockRelease(&sharedent->header.lock);
- pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
- pgstat_send(tsmsg, len);
+ return true;
}
-/*
- * Subroutine for pgstat_report_stat: populate and send a function stat message
- */
-static void
-pgstat_send_funcstats(void)
-{
- /* we assume this inits to all zeroes: */
- static const PgStat_FunctionCounts all_zeroes;
-
- PgStat_MsgFuncstat msg;
- PgStat_BackendFunctionEntry *entry;
- HASH_SEQ_STATUS fstat;
-
- if (pgStatFunctions == NULL)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_FUNCSTAT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_nentries = 0;
-
- hash_seq_init(&fstat, pgStatFunctions);
- while ((entry = (PgStat_BackendFunctionEntry *) hash_seq_search(&fstat)) != NULL)
- {
- PgStat_FunctionEntry *m_ent;
-
- /* Skip it if no counts accumulated since last time */
- if (memcmp(&entry->f_counts, &all_zeroes,
- sizeof(PgStat_FunctionCounts)) == 0)
- continue;
-
- /* need to convert format of time accumulators */
- m_ent = &msg.m_entry[msg.m_nentries];
- m_ent->f_id = entry->f_id;
- m_ent->f_numcalls = entry->f_counts.f_numcalls;
- m_ent->f_total_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_total_time);
- m_ent->f_self_time = INSTR_TIME_GET_MICROSEC(entry->f_counts.f_self_time);
-
- if (++msg.m_nentries >= PGSTAT_NUM_FUNCENTRIES)
- {
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
- msg.m_nentries = 0;
- }
-
- /* reset the entry's counts */
- MemSet(&entry->f_counts, 0, sizeof(PgStat_FunctionCounts));
- }
-
- if (msg.m_nentries > 0)
- pgstat_send(&msg, offsetof(PgStat_MsgFuncstat, m_entry[0]) +
- msg.m_nentries * sizeof(PgStat_FunctionEntry));
-
- have_function_stats = false;
-}
-
-
/* ----------
* pgstat_vacuum_stat() -
*
- * Will tell the collector about objects he can get rid of.
+ * Delete shared stat entries that are not in system catalogs.
+ *
+ * To avoid holding exclusive lock on dshash for a long time, the process is
+ * performed in three steps.
+ *
+ * 1: Collect existent oids of every kind of object.
+ * 2: Collect victim entries by scanning with shared lock.
+ * 3: Try removing every nominated entry without waiting for lock.
+ *
+ * As the consequence of the last step, some entries may be left alone due to
+ * lock failure, but as explained by the comment of pgstat_vacuum_stat, they
+ * will be deleted by later vacuums.
* ----------
*/
void
pgstat_vacuum_stat(void)
{
- HTAB *htab;
- PgStat_MsgTabpurge msg;
- PgStat_MsgFuncpurge f_msg;
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Read pg_database and make a list of OIDs of all existing databases
- */
- htab = pgstat_collect_oids(DatabaseRelationId, Anum_pg_database_oid);
-
- /*
- * Search the database hash table for dead databases and tell the
- * collector to drop them.
- */
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
- {
- Oid dbid = dbentry->databaseid;
-
- CHECK_FOR_INTERRUPTS();
-
- /* the DB entry for shared tables (with InvalidOid) is never dropped */
- if (OidIsValid(dbid) &&
- hash_search(htab, (void *) &dbid, HASH_FIND, NULL) == NULL)
- pgstat_drop_database(dbid);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Lookup our own database entry; if not found, nothing more to do.
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &MyDatabaseId,
- HASH_FIND, NULL);
- if (dbentry == NULL || dbentry->tables == NULL)
+ pgstat_oid_hash *dbids; /* database ids */
+ pgstat_oid_hash *relids; /* relation ids in the current database */
+ pgstat_oid_hash *funcids;/* function ids in the current database */
+ int nvictims = 0; /* # of entries of the above */
+ dshash_seq_status dshstat;
+ PgStatHashEntry *ent;
+
+ /* we don't collect stats under standalone mode */
+ if (!IsUnderPostmaster)
return;
- /*
- * Similarly to above, make a list of all known relations in this DB.
- */
- htab = pgstat_collect_oids(RelationRelationId, Anum_pg_class_oid);
+ /* collect oids of existent objects */
+ dbids = collect_oids(DatabaseRelationId, Anum_pg_database_oid);
+ relids = collect_oids(RelationRelationId, Anum_pg_class_oid);
+ funcids = collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
- /*
- * Initialize our messages table counter to zero
- */
- msg.m_nentries = 0;
+ nvictims = 0;
- /*
- * Check for all tables listed in stats hashtable if they still exist.
- */
- hash_seq_init(&hstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&hstat)) != NULL)
+ /* some of the dshash entries are to be removed, take exclusive lock. */
+ dshash_seq_init(&dshstat, pgStatSharedHash, true);
+ while ((ent = dshash_seq_next(&dshstat)) != NULL)
{
- Oid tabid = tabentry->tableid;
+ pgstat_oid_hash *oidhash;
+ Oid key;
CHECK_FOR_INTERRUPTS();
- if (hash_search(htab, (void *) &tabid, HASH_FIND, NULL) != NULL)
+ /*
+ * Don't drop entries for other than database objects not of the
+ * current database.
+ */
+ if (ent->key.type != PGSTAT_TYPE_DB &&
+ ent->key.databaseid != MyDatabaseId)
continue;
- /*
- * Not there, so add this table's Oid to the message
- */
- msg.m_tableid[msg.m_nentries++] = tabid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (msg.m_nentries >= PGSTAT_NUM_TABPURGE)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
-
- msg.m_nentries = 0;
- }
- }
-
- /*
- * Send the rest
- */
- if (msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0])
- + msg.m_nentries * sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
- }
-
- /* Clean up */
- hash_destroy(htab);
-
- /*
- * Now repeat the above steps for functions. However, we needn't bother
- * in the common case where no function stats are being collected.
- */
- if (dbentry->functions != NULL &&
- hash_get_num_entries(dbentry->functions) > 0)
- {
- htab = pgstat_collect_oids(ProcedureRelationId, Anum_pg_proc_oid);
-
- pgstat_setheader(&f_msg.m_hdr, PGSTAT_MTYPE_FUNCPURGE);
- f_msg.m_databaseid = MyDatabaseId;
- f_msg.m_nentries = 0;
-
- hash_seq_init(&hstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&hstat)) != NULL)
+ switch (ent->key.type)
{
- Oid funcid = funcentry->functionid;
-
- CHECK_FOR_INTERRUPTS();
-
- if (hash_search(htab, (void *) &funcid, HASH_FIND, NULL) != NULL)
+ case PGSTAT_TYPE_DB:
+ /* don't remove database entry for shared tables */
+ if (ent->key.databaseid == 0)
+ continue;
+ oidhash = dbids;
+ key = ent->key.databaseid;
+ break;
+
+ case PGSTAT_TYPE_TABLE:
+ oidhash = relids;
+ key = ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_FUNCTION:
+ oidhash = funcids;
+ key = ent->key.objectid;
+ break;
+
+ case PGSTAT_TYPE_REPLSLOT:
+ /*
+ * We don't bother vacuumming this kind of entries because the
+ * number of entries is quite small and entries are likely to
+ * be reused soon.
+ */
continue;
-
- /*
- * Not there, so add this function's Oid to the message
- */
- f_msg.m_functionid[f_msg.m_nentries++] = funcid;
-
- /*
- * If the message is full, send it out and reinitialize to empty
- */
- if (f_msg.m_nentries >= PGSTAT_NUM_FUNCPURGE)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
-
- f_msg.m_nentries = 0;
- }
}
- /*
- * Send the rest
- */
- if (f_msg.m_nentries > 0)
- {
- len = offsetof(PgStat_MsgFuncpurge, m_functionid[0])
- + f_msg.m_nentries * sizeof(Oid);
-
- pgstat_send(&f_msg, len);
- }
+ /* Skip existent objects. */
+ if (pgstat_oid_lookup(oidhash, key) != NULL)
+ continue;
- hash_destroy(htab);
+ /* drop this etnry */
+ delete_current_stats_entry(&dshstat);
+ nvictims++;
}
+ dshash_seq_term(&dshstat);
+ pgstat_oid_destroy(dbids);
+ pgstat_oid_destroy(relids);
+ pgstat_oid_destroy(funcids);
+
+ if (nvictims > 0)
+ pg_atomic_add_fetch_u64(&StatsShmem->gc_count, 1);
}
-
/* ----------
- * pgstat_collect_oids() -
+ * collect_oids() -
*
* Collect the OIDs of all objects listed in the specified system catalog
- * into a temporary hash table. Caller should hash_destroy the result
+ * into a temporary hash table. Caller should pgsstat_oid_destroy the result
* when done with it. (However, we make the table in CurrentMemoryContext
* so that it will be freed properly in event of an error.)
* ----------
*/
-static HTAB *
-pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
+static pgstat_oid_hash *
+collect_oids(Oid catalogid, AttrNumber anum_oid)
{
- HTAB *htab;
- HASHCTL hash_ctl;
+ pgstat_oid_hash *rethash;
Relation rel;
TableScanDesc scan;
HeapTuple tup;
Snapshot snapshot;
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(Oid);
- hash_ctl.hcxt = CurrentMemoryContext;
- htab = hash_create("Temporary table of OIDs",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ rethash = pgstat_oid_create(CurrentMemoryContext,
+ PGSTAT_TABLE_HASH_SIZE, NULL);
rel = table_open(catalogid, AccessShareLock);
snapshot = RegisterSnapshot(GetLatestSnapshot());
@@ -1280,81 +1678,61 @@ pgstat_collect_oids(Oid catalogid, AttrNumber anum_oid)
{
Oid thisoid;
bool isnull;
+ bool found;
thisoid = heap_getattr(tup, anum_oid, RelationGetDescr(rel), &isnull);
Assert(!isnull);
CHECK_FOR_INTERRUPTS();
- (void) hash_search(htab, (void *) &thisoid, HASH_ENTER, NULL);
+ pgstat_oid_insert(rethash, thisoid, &found);
}
table_endscan(scan);
UnregisterSnapshot(snapshot);
table_close(rel, AccessShareLock);
- return htab;
+ return rethash;
}
-
/* ----------
* pgstat_drop_database() -
*
- * Tell the collector that we just dropped a database.
- * (If the message gets lost, we will still clean the dead DB eventually
- * via future invocations of pgstat_vacuum_stat().)
- * ----------
+ * Remove entry for the database that we just dropped.
+ *
+ * Some entries might be left alone due to lock failure or some stats are
+ * flushed after this but we will still clean the dead DB eventually via
+ * future invocations of pgstat_vacuum_stat().
+ * ----------
*/
void
pgstat_drop_database(Oid databaseid)
{
- PgStat_MsgDropdb msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DROPDB);
- msg.m_databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
+ Assert(OidIsValid(databaseid));
-/* ----------
- * pgstat_drop_relation() -
- *
- * Tell the collector that we just dropped a relation.
- * (If the message gets lost, we will still clean the dead entry eventually
- * via future invocations of pgstat_vacuum_stat().)
- *
- * Currently not used for lack of any good place to call it; we rely
- * entirely on pgstat_vacuum_stat() to clean out stats for dead rels.
- * ----------
- */
-#ifdef NOT_USED
-void
-pgstat_drop_relation(Oid relid)
-{
- PgStat_MsgTabpurge msg;
- int len;
-
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- msg.m_tableid[0] = relid;
- msg.m_nentries = 1;
-
- len = offsetof(PgStat_MsgTabpurge, m_tableid[0]) + sizeof(Oid);
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TABPURGE);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, len);
+ /* some of the dshash entries are to be removed, take exclusive lock. */
+ dshash_seq_init(&hstat, pgStatSharedHash, true);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.databaseid == MyDatabaseId)
+ delete_current_stats_entry(&hstat);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Let readers run a garbage collection of local hashes */
+ pg_atomic_add_fetch_u64(&StatsShmem->gc_count, 1);
}
-#endif /* NOT_USED */
/* ----------
* pgstat_reset_counters() -
*
- * Tell the statistics collector to reset counters for our database.
+ * Reset counters for our database.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1363,53 +1741,146 @@ pgstat_drop_relation(Oid relid)
void
pgstat_reset_counters(void)
{
- PgStat_MsgResetcounter msg;
+ dshash_seq_status hstat;
+ PgStatHashEntry *p;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETCOUNTER);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatEntryHeader *header;
+
+ if (p->key.databaseid != MyDatabaseId)
+ continue;
+
+ header = dsa_get_address(area, p->body);
+
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ memset(PGSTAT_SHENT_BODY(header), 0,
+ PGSTAT_SHENT_BODY_LEN(p->key.type));
+
+ if (p->key.type == PGSTAT_TYPE_DB)
+ {
+ PgStat_StatDBEntry *dbstat = (PgStat_StatDBEntry *) header;
+ dbstat->stat_reset_timestamp = GetCurrentTimestamp();
+ }
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Invalidate the simple cache keys */
+ cached_dbent_key = stathashkey_zero;
+ cached_tabent_key = stathashkey_zero;
+ cached_funcent_key = stathashkey_zero;
+}
+
+/*
+ * pgstat_copy_global_stats - helper function for functions
+ * pgstat_fetch_stat_*() and pgstat_reset_shared_counters().
+ *
+ * Copies out the specified memory area following change-count protocol.
+ */
+static inline void
+pgstat_copy_global_stats(void *dst, void*src, size_t len,
+ pg_atomic_uint32 *count)
+{
+ int before_changecount;
+ int after_changecount;
+
+ after_changecount = pg_atomic_read_u32(count);
+
+ do
+ {
+ before_changecount = after_changecount;
+ memcpy(dst, src, len);
+ after_changecount = pg_atomic_read_u32(count);
+ } while ((before_changecount & 1) == 1 ||
+ after_changecount != before_changecount);
}
/* ----------
* pgstat_reset_shared_counters() -
*
- * Tell the statistics collector to reset cluster-wide shared counters.
+ * Reset cluster-wide shared counters.
*
* Permission checking for this function is managed through the normal
* GRANT system.
+ *
+ * We don't scribble on shared stats while resetting to avoid locking on
+ * shared stats struct. Instead, just record the current counters in another
+ * shared struct, which is protected by StatsLock. See
+ * pgstat_fetch_stat_(archiver|bgwriter|checkpointer) for the reader side.
* ----------
*/
void
pgstat_reset_shared_counters(const char *target)
{
- PgStat_MsgResetsharedcounter msg;
-
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ TimestampTz now = GetCurrentTimestamp();
+ PgStat_Shared_Reset_Target t;
if (strcmp(target, "archiver") == 0)
- msg.m_resettarget = RESET_ARCHIVER;
+ t = RESET_ARCHIVER;
else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
+ t = RESET_BGWRITER;
else if (strcmp(target, "wal") == 0)
- msg.m_resettarget = RESET_WAL;
+ t = RESET_WAL;
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
errhint("Target must be \"archiver\", \"bgwriter\" or \"wal\".")));
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
- pgstat_send(&msg, sizeof(msg));
+ /* Reset statistics for the cluster. */
+ LWLockAcquire(StatsLock, LW_EXCLUSIVE);
+
+ switch (t)
+ {
+ case RESET_ARCHIVER:
+ pgstat_copy_global_stats(&StatsShmem->archiver_reset_offset,
+ &StatsShmem->archiver_stats,
+ sizeof(PgStat_Archiver),
+ &StatsShmem->archiver_changecount);
+ StatsShmem->archiver_reset_offset.stat_reset_timestamp = now;
+ cached_archiverstats_is_valid = false;
+ break;
+
+ case RESET_BGWRITER:
+ pgstat_copy_global_stats(&StatsShmem->bgwriter_reset_offset,
+ &StatsShmem->bgwriter_stats,
+ sizeof(PgStat_BgWriter),
+ &StatsShmem->bgwriter_changecount);
+ pgstat_copy_global_stats(&StatsShmem->checkpointer_reset_offset,
+ &StatsShmem->checkpointer_stats,
+ sizeof(PgStat_CheckPointer),
+ &StatsShmem->checkpointer_changecount);
+ StatsShmem->bgwriter_reset_offset.stat_reset_timestamp = now;
+ cached_bgwriterstats_is_valid = false;
+ cached_checkpointerstats_is_valid = false;
+ break;
+
+ case RESET_WAL:
+ /*
+ * Differntly from the two above, WAL statistics has many writer
+ * processes and protected by wal_stats_lock.
+ */
+ LWLockAcquire(&StatsShmem->wal_stats_lock, LW_EXCLUSIVE);
+ MemSet(&StatsShmem->wal_stats, 0, sizeof(PgStat_Wal));
+ StatsShmem->wal_stats.stat_reset_timestamp = now;
+ LWLockRelease(&StatsShmem->wal_stats_lock);
+ cached_walstats_is_valid = false;
+ break;
+ }
+
+ LWLockRelease(StatsLock);
}
/* ----------
* pgstat_reset_single_counter() -
*
- * Tell the statistics collector to reset a single counter.
+ * Reset a single counter.
*
* Permission checking for this function is managed through the normal
* GRANT system.
@@ -1418,17 +1889,37 @@ pgstat_reset_shared_counters(const char *target)
void
pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
{
- PgStat_MsgResetsinglecounter msg;
+ PgStat_StatEntryHeader *header;
+ PgStat_StatDBEntry *dbentry;
+ PgStatTypes stattype;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ dbentry = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, MyDatabaseId, InvalidOid, false, false,
+ NULL);
+ Assert(dbentry);
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSINGLECOUNTER);
- msg.m_databaseid = MyDatabaseId;
- msg.m_resettype = type;
- msg.m_objectid = objoid;
+ /* Set the reset timestamp for the whole database */
+ ts = GetCurrentTimestamp();
+ LWLockAcquire(&dbentry->header.lock, LW_EXCLUSIVE);
+ dbentry->stat_reset_timestamp = ts;
+ LWLockRelease(&dbentry->header.lock);
- pgstat_send(&msg, sizeof(msg));
+ /* Remove object if it exists, ignore if not */
+ switch (type)
+ {
+ case RESET_TABLE:
+ stattype = PGSTAT_TYPE_TABLE;
+ break;
+ case RESET_FUNCTION:
+ stattype = PGSTAT_TYPE_FUNCTION;
+ }
+
+ header = get_stat_entry(stattype, MyDatabaseId, objoid, false, false, NULL);
+
+ LWLockAcquire(&header->lock, LW_EXCLUSIVE);
+ memset(PGSTAT_SHENT_BODY(header), 0, PGSTAT_SHENT_BODY_LEN(stattype));
+ LWLockRelease(&header->lock);
}
/* ----------
@@ -1444,15 +1935,40 @@ pgstat_reset_single_counter(Oid objoid, PgStat_Single_Reset_Type type)
void
pgstat_reset_slru_counter(const char *name)
{
- PgStat_MsgResetslrucounter msg;
+ int i;
+ TimestampTz ts = GetCurrentTimestamp();
+ uint32 assert_changecount;PG_USED_FOR_ASSERTS_ONLY;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ if (name)
+ {
+ i = pgstat_slru_index(name);
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_changecount, 1);
+ Assert((assert_changecount & 1) == 0);
+ MemSet(&StatsShmem->slru_stats.entry[i], 0,
+ sizeof(PgStat_SLRUStats));
+ StatsShmem->slru_stats.entry[i].stat_reset_timestamp = ts;
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+ }
+ else
+ {
+ LWLockAcquire(&StatsShmem->slru_stats.lock, LW_EXCLUSIVE);
+ assert_changecount =
+ pg_atomic_fetch_add_u32(&StatsShmem->slru_changecount, 1);
+ Assert((assert_changecount & 1) == 0);
+ for (i = 0 ; i < SLRU_NUM_ELEMENTS; i++)
+ {
+ MemSet(&StatsShmem->slru_stats.entry[i], 0,
+ sizeof(PgStat_SLRUStats));
+ StatsShmem->slru_stats.entry[i].stat_reset_timestamp = ts;
+ }
+ pg_atomic_add_fetch_u32(&StatsShmem->slru_changecount, 1);
+ LWLockRelease(&StatsShmem->slru_stats.lock);
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSLRUCOUNTER);
- msg.m_index = (name) ? pgstat_slru_index(name) : -1;
-
- pgstat_send(&msg, sizeof(msg));
+ cached_slrustats_is_valid = false;
}
/* ----------
@@ -1468,20 +1984,19 @@ pgstat_reset_slru_counter(const char *name)
void
pgstat_reset_replslot_counter(const char *name)
{
- PgStat_MsgResetreplslotcounter msg;
+ int startidx;
+ int endidx;
+ int i;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ if (!IsUnderPostmaster || !pgStatSharedHash)
return;
if (name)
{
ReplicationSlot *slot;
-
- /*
- * Check if the slot exits with the given name. It is possible that by
- * the time this message is executed the slot is dropped but at least
- * this check will ensure that the given name is for a valid slot.
- */
+
+ /* Check if the slot exits with the given name. */
LWLockAcquire(ReplicationSlotControlLock, LW_SHARED);
slot = SearchNamedReplicationSlot(name);
LWLockRelease(ReplicationSlotControlLock);
@@ -1499,15 +2014,36 @@ pgstat_reset_replslot_counter(const char *name)
if (SlotIsPhysical(slot))
return;
- strlcpy(msg.m_slotname, name, NAMEDATALEN);
- msg.clearall = false;
+ /* reset this one entry */
+ startidx = endidx = slot - ReplicationSlotCtl->replication_slots;
}
else
- msg.clearall = true;
+ {
+ /* reset all existent entries */
+ startidx = 0;
+ endidx = max_replication_slots - 1;
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETREPLSLOTCOUNTER);
+ ts = GetCurrentTimestamp();
+ for (i = startidx ; i <= endidx ; i++)
+ {
+ PgStat_ReplSlot *shent;
- pgstat_send(&msg, sizeof(msg));
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, false, NULL);
+
+ /* Skip non-existent entries */
+ if (!shent)
+ continue;
+
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ memset(&shent->spill_txns, 0,
+ offsetof(PgStat_ReplSlot, stat_reset_timestamp) -
+ offsetof(PgStat_ReplSlot, spill_txns));
+ shent->stat_reset_timestamp = ts;
+ LWLockRelease(&shent->header.lock);
+ }
}
/* ----------
@@ -1521,48 +2057,93 @@ pgstat_reset_replslot_counter(const char *name)
void
pgstat_report_autovac(Oid dboid)
{
- PgStat_MsgAutovacStart msg;
+ PgStat_StatDBEntry *dbentry;
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET)
+ /* return if activity stats is not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_AUTOVAC_START);
- msg.m_databaseid = dboid;
- msg.m_start_time = GetCurrentTimestamp();
+ /*
+ * End-of-vacuum is reported instantly. Report the start the same way for
+ * consistency. Vacuum doesn't run frequently and is a long-lasting
+ * operation so it doesn't matter if we get blocked here a little.
+ */
+ dbentry = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, dboid, InvalidOid, false, true, NULL);
- pgstat_send(&msg, sizeof(msg));
+ ts = GetCurrentTimestamp();;
+ LWLockAcquire(&dbentry->header.lock, LW_EXCLUSIVE);
+ dbentry->last_autovac_time = ts;
+ LWLockRelease(&dbentry->header.lock);
}
/* ---------
* pgstat_report_vacuum() -
*
- * Tell the collector about the table we just vacuumed.
+ * Report about the table we just vacuumed.
* ---------
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples)
{
- PgStat_MsgVacuum msg;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+ TimestampTz ts;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
- msg.m_databaseid = shared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = tableoid;
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_vacuumtime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /* Store the data in the table's hash table entry. */
+ ts = GetCurrentTimestamp();
+
+ /*
+ * Differently from ordinary operations, maintenance commands take longer
+ * time and getting blocked at the end of work doesn't matter. Furthermore,
+ * this can prevent the stats updates made by the transactions that ends
+ * after this vacuum from being canceled by a delayed vacuum report.
+ * Update shared stats entry directly for the above reasons.
+ */
+ tabentry = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, tableoid, false, true, NULL);
+
+ LWLockAcquire(&tabentry->header.lock, LW_EXCLUSIVE);
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * It is quite possible that a non-aggressive VACUUM ended up skipping
+ * various pages, however, we'll zero the insert counter here regardless.
+ * It's currently used only to track when we need to perform an "insert"
+ * autovacuum, which are mainly intended to freeze newly inserted tuples.
+ * Zeroing this may just mean we'll not try to vacuum the table again
+ * until enough tuples have been inserted to trigger another insert
+ * autovacuum. An anti-wraparound autovacuum will catch any persistent
+ * stragglers.
+ */
+ tabentry->inserts_since_vacuum = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_vacuum_timestamp = ts;
+ tabentry->autovac_vacuum_count++;
+ }
+ else
+ {
+ tabentry->vacuum_timestamp = ts;
+ tabentry->vacuum_count++;
+ }
+
+ LWLockRelease(&tabentry->header.lock);
}
/* --------
* pgstat_report_analyze() -
*
- * Tell the collector about the table we just analyzed.
+ * Report about the table we just analyzed.
*
* Caller must provide new live- and dead-tuples estimates, as well as a
* flag indicating whether to reset the changes_since_analyze counter.
@@ -1573,9 +2154,11 @@ pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter)
{
- PgStat_MsgAnalyze msg;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId);
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
/*
@@ -1583,10 +2166,10 @@ pgstat_report_analyze(Relation rel,
* already inserted and/or deleted rows in the target table. ANALYZE will
* have counted such rows as live or dead respectively. Because we will
* report our counts of such rows at transaction end, we should subtract
- * off these counts from what we send to the collector now, else they'll
- * be double-counted after commit. (This approach also ensures that the
- * collector ends up with the right numbers if we abort instead of
- * committing.)
+ * off these counts from what is already written to shared stats now, else
+ * they'll be double-counted after commit. (This approach also ensures
+ * that the shared stats ends up with the right numbers if we abort
+ * instead of committing.)
*/
if (rel->pgstat_info != NULL)
{
@@ -1604,137 +2187,223 @@ pgstat_report_analyze(Relation rel,
deadtuples = Max(deadtuples, 0);
}
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
- msg.m_databaseid = rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId;
- msg.m_tableoid = RelationGetRelid(rel);
- msg.m_autovacuum = IsAutoVacuumWorkerProcess();
- msg.m_resetcounter = resetcounter;
- msg.m_analyzetime = GetCurrentTimestamp();
- msg.m_live_tuples = livetuples;
- msg.m_dead_tuples = deadtuples;
- pgstat_send(&msg, sizeof(msg));
+ /*
+ * Differently from ordinary operations, maintenance commands take longer
+ * time and getting blocked at the end of work doesn't matter. Furthermore,
+ * this can prevent the stats updates made by the transactions that ends
+ * after this analyze from being canceled by a delayed analyze report.
+ * Update shared stats entry directly for the above reasons.
+ */
+ tabentry = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, RelationGetRelid(rel),
+ false, true, NULL);
+
+ LWLockAcquire(&tabentry->header.lock, LW_EXCLUSIVE);
+ tabentry->n_live_tuples = livetuples;
+ tabentry->n_dead_tuples = deadtuples;
+
+ /*
+ * If commanded, reset changes_since_analyze to zero. This forgets any
+ * changes that were committed while the ANALYZE was in progress, but we
+ * have no good way to estimate how many of those there were.
+ */
+ if (resetcounter)
+ tabentry->changes_since_analyze = 0;
+
+ if (IsAutoVacuumWorkerProcess())
+ {
+ tabentry->autovac_analyze_timestamp = GetCurrentTimestamp();
+ tabentry->autovac_analyze_count++;
+ }
+ else
+ {
+ tabentry->analyze_timestamp = GetCurrentTimestamp();
+ tabentry->analyze_count++;
+ }
+ LWLockRelease(&tabentry->header.lock);
}
/* --------
* pgstat_report_recovery_conflict() -
*
- * Tell the collector about a Hot Standby recovery conflict.
+ * Report a Hot Standby recovery conflict.
* --------
*/
void
pgstat_report_recovery_conflict(int reason)
{
- PgStat_MsgRecoveryConflict msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RECOVERYCONFLICT);
- msg.m_databaseid = MyDatabaseId;
- msg.m_reason = reason;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+
+ switch (reason)
+ {
+ case PROCSIG_RECOVERY_CONFLICT_DATABASE:
+
+ /*
+ * Since we drop the information about the database as soon as it
+ * replicates, there is no point in counting these conflicts.
+ */
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
+ dbent->counts.n_conflict_tablespace++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_LOCK:
+ dbent->counts.n_conflict_lock++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
+ dbent->counts.n_conflict_snapshot++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
+ dbent->counts.n_conflict_bufferpin++;
+ break;
+ case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
+ dbent->counts.n_conflict_startup_deadlock++;
+ break;
+ }
}
+
/* --------
* pgstat_report_deadlock() -
*
- * Tell the collector about a deadlock detected.
+ * Report a deadlock detected.
* --------
*/
void
pgstat_report_deadlock(void)
{
- PgStat_MsgDeadlock msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DEADLOCK);
- msg.m_databaseid = MyDatabaseId;
- pgstat_send(&msg, sizeof(msg));
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_deadlocks++;
}
-
-
/* --------
- * pgstat_report_checksum_failures_in_db() -
+ * pgstat_report_checksum_failures_in_db(dboid, failure_count) -
*
- * Tell the collector about one or more checksum failures.
+ * Reports about one or more checksum failures.
* --------
*/
void
pgstat_report_checksum_failures_in_db(Oid dboid, int failurecount)
{
- PgStat_MsgChecksumFailure msg;
+ PgStat_StatDBEntry *dbentry;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not active */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_CHECKSUMFAILURE);
- msg.m_databaseid = dboid;
- msg.m_failurecount = failurecount;
- msg.m_failure_time = GetCurrentTimestamp();
+ dbentry = get_local_dbstat_entry(dboid);
- pgstat_send(&msg, sizeof(msg));
+ /* add accumulated count to the parameter */
+ dbentry->counts.n_checksum_failures += failurecount;
}
/* --------
* pgstat_report_checksum_failure() -
*
- * Tell the collector about a checksum failure.
+ * Reports about a checksum failure.
* --------
*/
void
pgstat_report_checksum_failure(void)
{
- pgstat_report_checksum_failures_in_db(MyDatabaseId, 1);
+ PgStat_StatDBEntry *dbent;
+
+ /* return if we are not collecting stats */
+ if (!area)
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_checksum_failures++;
}
/* --------
* pgstat_report_tempfile() -
*
- * Tell the collector about a temporary file.
+ * Report a temporary file.
* --------
*/
void
pgstat_report_tempfile(size_t filesize)
{
- PgStat_MsgTempFile msg;
+ PgStat_StatDBEntry *dbent;
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
return;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_TEMPFILE);
- msg.m_databaseid = MyDatabaseId;
- msg.m_filesize = filesize;
- pgstat_send(&msg, sizeof(msg));
+ if (filesize == 0) /* Is there a case where filesize is really 0? */
+ return;
+
+ dbent = get_local_dbstat_entry(MyDatabaseId);
+ dbent->counts.n_temp_bytes += filesize; /* needs check overflow */
+ dbent->counts.n_temp_files++;
}
/* ----------
* pgstat_report_replslot() -
*
- * Tell the collector about replication slot statistics.
+ * Report replication slot activity.
* ----------
*/
void
-pgstat_report_replslot(const char *slotname, int spilltxns, int spillcount,
- int spillbytes, int streamtxns, int streamcount, int streambytes)
+pgstat_report_replslot(const char *slotname,
+ int spilltxns, int spillcount, int spillbytes,
+ int streamtxns, int streamcount, int streambytes)
{
- PgStat_MsgReplSlot msg;
+ PgStat_ReplSlot *shent;
+ int i;
+ bool found;
+
+ if (!area)
+ return;
+
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ if (strcmp(NameStr(ReplicationSlotCtl->replication_slots[i].data.name),
+ slotname) == 0)
+ break;
+
+ }
/*
- * Prepare and send the message
+ * the slot should have been removed. just ignore it. We create the entry
+ * for the slot with this name next time.
*/
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_REPLSLOT);
- strlcpy(msg.m_slotname, slotname, NAMEDATALEN);
- msg.m_drop = false;
- msg.m_spill_txns = spilltxns;
- msg.m_spill_count = spillcount;
- msg.m_spill_bytes = spillbytes;
- msg.m_stream_txns = streamtxns;
- msg.m_stream_count = streamcount;
- msg.m_stream_bytes = streambytes;
- pgstat_send(&msg, sizeof(PgStat_MsgReplSlot));
+ if (i == max_replication_slots)
+ return;
+
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, true, &found);
+
+ /* Clear the counters and reset dropped when we reuse it */
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ if (shent->header.dropped || !found)
+ {
+ memset(&shent->spill_txns, 0,
+ sizeof(PgStat_ReplSlot) - offsetof(PgStat_ReplSlot, spill_txns));
+ strlcpy(shent->slotname, slotname, NAMEDATALEN);
+ shent->header.dropped = false;
+ }
+
+ shent->spill_txns += spilltxns;
+ shent->spill_count += spillcount;
+ shent->spill_bytes += spillbytes;
+ shent->stream_txns += streamtxns;
+ shent->stream_count += streamcount;
+ shent->stream_bytes += streambytes;
+ LWLockRelease(&shent->header.lock);
}
/* ----------
@@ -1746,55 +2415,44 @@ pgstat_report_replslot(const char *slotname, int spilltxns, int spillcount,
void
pgstat_report_replslot_drop(const char *slotname)
{
- PgStat_MsgReplSlot msg;
+ int i;
+ PgStat_ReplSlot *shent;
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_REPLSLOT);
- strlcpy(msg.m_slotname, slotname, NAMEDATALEN);
- msg.m_drop = true;
- pgstat_send(&msg, sizeof(PgStat_MsgReplSlot));
-}
+ Assert(area);
+ if (!area)
+ return;
-/* ----------
- * pgstat_ping() -
- *
- * Send some junk data to the collector to increase traffic.
- * ----------
- */
-void
-pgstat_ping(void)
-{
- PgStat_MsgDummy msg;
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ if (strcmp(NameStr(ReplicationSlotCtl->replication_slots[i].data.name),
+ slotname) == 0)
+ break;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
+ }
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_DUMMY);
- pgstat_send(&msg, sizeof(msg));
+ /* XXX: maybe the slot has been removed. just ignore it. */
+ if (i == max_replication_slots)
+ return;
+
+ shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT,
+ MyDatabaseId, i, false, false, NULL);
+
+ if (shent && !shent->header.dropped)
+ {
+ LWLockAcquire(&shent->header.lock, LW_EXCLUSIVE);
+ shent->header.dropped = true;
+ LWLockRelease(&shent->header.lock);
+ }
}
/* ----------
- * pgstat_send_inquiry() -
+ * pgstat_init_function_usage() -
*
- * Notify collector that we need fresh data.
+ * Initialize function call usage data.
+ * Called by the executor before invoking a function.
* ----------
*/
-static void
-pgstat_send_inquiry(TimestampTz clock_time, TimestampTz cutoff_time, Oid databaseid)
-{
- PgStat_MsgInquiry msg;
-
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_INQUIRY);
- msg.clock_time = clock_time;
- msg.cutoff_time = cutoff_time;
- msg.databaseid = databaseid;
- pgstat_send(&msg, sizeof(msg));
-}
-
-
-/*
- * Initialize function call usage data.
- * Called by the executor before invoking a function.
- */
void
pgstat_init_function_usage(FunctionCallInfo fcinfo,
PgStat_FunctionCallUsage *fcu)
@@ -1809,24 +2467,9 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
return;
}
- if (!pgStatFunctions)
- {
- /* First time through - initialize function stat table */
- HASHCTL hash_ctl;
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_BackendFunctionEntry);
- pgStatFunctions = hash_create("Function stat entries",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
- }
-
- /* Get the stats entry for this function, create if necessary */
- htabent = hash_search(pgStatFunctions, &fcinfo->flinfo->fn_oid,
- HASH_ENTER, &found);
- if (!found)
- MemSet(&htabent->f_counts, 0, sizeof(PgStat_FunctionCounts));
+ htabent = (PgStat_BackendFunctionEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ fcinfo->flinfo->fn_oid, true, &found);
fcu->fs = &htabent->f_counts;
@@ -1840,31 +2483,37 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
INSTR_TIME_SET_CURRENT(fcu->f_start);
}
-/*
- * find_funcstat_entry - find any existing PgStat_BackendFunctionEntry entry
- * for specified function
+/* ----------
+ * find_funcstat_entry() -
*
- * If no entry, return NULL, don't create a new one
+ * find any existing PgStat_BackendFunctionEntry entry for specified function
+ *
+ * If no entry, return NULL, not creating a new one.
+ * ----------
*/
PgStat_BackendFunctionEntry *
find_funcstat_entry(Oid func_id)
{
- if (pgStatFunctions == NULL)
- return NULL;
+ PgStat_BackendFunctionEntry *ent;
- return (PgStat_BackendFunctionEntry *) hash_search(pgStatFunctions,
- (void *) &func_id,
- HASH_FIND, NULL);
+ ent = (PgStat_BackendFunctionEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_FUNCTION, MyDatabaseId,
+ func_id, false, NULL);
+
+ return ent;
}
-/*
- * Calculate function call usage and update stat counters.
- * Called by the executor after invoking a function.
+/* ----------
+ * pgstat_end_function_usage() -
*
- * In the case of a set-returning function that runs in value-per-call mode,
- * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
- * calls for what the user considers a single call of the function. The
- * finalize flag should be TRUE on the last call.
+ * Calculate function call usage and update stat counters.
+ * Called by the executor after invoking a function.
+ *
+ * In the case of a set-returning function that runs in value-per-call mode,
+ * we will see multiple pgstat_init_function_usage/pgstat_end_function_usage
+ * calls for what the user considers a single call of the function. The
+ * finalize flag should be TRUE on the last call.
+ * ----------
*/
void
pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
@@ -1905,9 +2554,6 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
fs->f_numcalls++;
fs->f_total_time = f_total;
INSTR_TIME_ADD(fs->f_self_time, f_self);
-
- /* indicate that we have something to send */
- have_function_stats = true;
}
@@ -1919,8 +2565,7 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
*
* We assume that a relcache entry's pgstat_info field is zeroed by
* relcache.c when the relcache entry is made; thereafter it is long-lived
- * data. We can avoid repeated searches of the TabStatus arrays when the
- * same relation is touched repeatedly within a transaction.
+ * data.
* ----------
*/
void
@@ -1936,7 +2581,8 @@ pgstat_initstats(Relation rel)
return;
}
- if (pgStatSock == PGINVALID_SOCKET || !pgstat_track_counts)
+ /* return if we are not collecting stats */
+ if (!area)
{
/* We're not counting at all */
rel->pgstat_info = NULL;
@@ -1947,120 +2593,60 @@ pgstat_initstats(Relation rel)
* If we already set up this relation in the current transaction, nothing
* to do.
*/
- if (rel->pgstat_info != NULL &&
- rel->pgstat_info->t_id == rel_id)
+ if (rel->pgstat_info != NULL)
return;
/* Else find or make the PgStat_TableStatus entry, and update link */
- rel->pgstat_info = get_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ rel->pgstat_info = get_local_tabstat_entry(rel_id, rel->rd_rel->relisshared);
+ /* mark this relation as the owner */
+
+ /* don't allow link a stats to multiple relcache entries */
+ Assert (rel->pgstat_info->relation == NULL);
+ rel->pgstat_info->relation = rel;
}
/*
- * get_tabstat_entry - find or create a PgStat_TableStatus entry for rel
+ * pgstat_delinkstats() -
+ *
+ * Break the mutual link between a relcache entry and a local stats entry.
+ * This must be called always when one end of the link is removed.
*/
-static PgStat_TableStatus *
-get_tabstat_entry(Oid rel_id, bool isshared)
+void
+pgstat_delinkstats(Relation rel)
{
- TabStatHashEntry *hash_entry;
- PgStat_TableStatus *entry;
- TabStatusArray *tsa;
- bool found;
-
- /*
- * Create hash table if we don't have it already.
- */
- if (pgStatTabHash == NULL)
+ /* remove the link to stats info if any */
+ if (rel && rel->pgstat_info)
{
- HASHCTL ctl;
-
- ctl.keysize = sizeof(Oid);
- ctl.entrysize = sizeof(TabStatHashEntry);
-
- pgStatTabHash = hash_create("pgstat TabStatusArray lookup hash table",
- TABSTAT_QUANTUM,
- &ctl,
- HASH_ELEM | HASH_BLOBS);
+ /* ilnk sanity check */
+ Assert (rel->pgstat_info->relation == rel);
+ rel->pgstat_info->relation = NULL;
+ rel->pgstat_info = NULL;
}
-
- /*
- * Find an entry or create a new one.
- */
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_ENTER, &found);
- if (!found)
- {
- /* initialize new entry with null pointer */
- hash_entry->tsa_entry = NULL;
- }
-
- /*
- * If entry is already valid, we're done.
- */
- if (hash_entry->tsa_entry)
- return hash_entry->tsa_entry;
-
- /*
- * Locate the first pgStatTabList entry with free space, making a new list
- * entry if needed. Note that we could get an OOM failure here, but if so
- * we have left the hashtable and the list in a consistent state.
- */
- if (pgStatTabList == NULL)
- {
- /* Set up first pgStatTabList entry */
- pgStatTabList = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- }
-
- tsa = pgStatTabList;
- while (tsa->tsa_used >= TABSTAT_QUANTUM)
- {
- if (tsa->tsa_next == NULL)
- tsa->tsa_next = (TabStatusArray *)
- MemoryContextAllocZero(TopMemoryContext,
- sizeof(TabStatusArray));
- tsa = tsa->tsa_next;
- }
-
- /*
- * Allocate a PgStat_TableStatus entry within this list entry. We assume
- * the entry was already zeroed, either at creation or after last use.
- */
- entry = &tsa->tsa_entries[tsa->tsa_used++];
- entry->t_id = rel_id;
- entry->t_shared = isshared;
-
- /*
- * Now we can fill the entry in pgStatTabHash.
- */
- hash_entry->tsa_entry = entry;
-
- return entry;
}
/*
* find_tabstat_entry - find any existing PgStat_TableStatus entry for rel
*
- * If no entry, return NULL, don't create a new one
+ * Find any existing PgStat_TableStatus entry for rel_id in the current
+ * database. If not found, try finding from shared tables.
*
- * Note: if we got an error in the most recent execution of pgstat_report_stat,
- * it's possible that an entry exists but there's no hashtable entry for it.
- * That's okay, we'll treat this case as "doesn't exist".
+ * If no entry found, return NULL, don't create a new one
+ * ----------
*/
PgStat_TableStatus *
find_tabstat_entry(Oid rel_id)
{
- TabStatHashEntry *hash_entry;
+ PgStat_TableStatus *ent;
- /* If hashtable doesn't exist, there are no entries at all */
- if (!pgStatTabHash)
- return NULL;
+ ent = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE, MyDatabaseId, rel_id,
+ false, NULL);
+ if (!ent)
+ ent = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE, InvalidOid, rel_id,
+ false, NULL);
- hash_entry = hash_search(pgStatTabHash, &rel_id, HASH_FIND, NULL);
- if (!hash_entry)
- return NULL;
-
- /* Note that this step could also return NULL, but that's correct */
- return hash_entry->tsa_entry;
+ return ent;
}
/*
@@ -2475,8 +3061,6 @@ AtPrepare_PgStat(void)
record.inserted_pre_trunc = trans->inserted_pre_trunc;
record.updated_pre_trunc = trans->updated_pre_trunc;
record.deleted_pre_trunc = trans->deleted_pre_trunc;
- record.t_id = tabstat->t_id;
- record.t_shared = tabstat->t_shared;
record.t_truncated = trans->truncated;
RegisterTwoPhaseRecord(TWOPHASE_RM_PGSTAT_ID, 0,
@@ -2491,8 +3075,8 @@ AtPrepare_PgStat(void)
*
* All we need do here is unlink the transaction stats state from the
* nontransactional state. The nontransactional action counts will be
- * reported to the stats collector immediately, while the effects on live
- * and dead tuple counts are preserved in the 2PC state file.
+ * reported to the activity stats facility immediately, while the effects on
+ * live and dead tuple counts are preserved in the 2PC state file.
*
* Note: AtEOXact_PgStat is not called during PREPARE.
*/
@@ -2537,7 +3121,7 @@ pgstat_twophase_postcommit(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, commit case */
pgstat_info->t_counts.t_tuples_inserted += rec->tuples_inserted;
@@ -2573,7 +3157,7 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
PgStat_TableStatus *pgstat_info;
/* Find or create a tabstat entry for the rel */
- pgstat_info = get_tabstat_entry(rec->t_id, rec->t_shared);
+ pgstat_info = get_local_tabstat_entry(rec->t_id, rec->t_shared);
/* Same math as in AtEOXact_PgStat, abort case */
if (rec->t_truncated)
@@ -2593,85 +3177,138 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
/* ----------
* pgstat_fetch_stat_dbentry() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one database or NULL. NULL doesn't mean
- * that the database doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * Find database stats entry on backends in a palloc'ed memory.
+ *
+ * The returned entry is stored in static memory so the content is valid until
+ * the next call of the same function for the different database.
* ----------
*/
PgStat_StatDBEntry *
pgstat_fetch_stat_dbentry(Oid dbid)
{
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup the requested database; return NULL if not found
- */
- return (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
+ PgStat_StatDBEntry *shent;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for InvalidOid */
+ Assert(dbid != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_dbent_key.databaseid == dbid)
+ return &cached_dbent;
+
+ shent = (PgStat_StatDBEntry *)
+ get_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid, true, false, NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_dbent, shent, sizeof(PgStat_StatDBEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_dbent_key.databaseid = dbid;
+
+ return &cached_dbent;
}
-
/* ----------
* pgstat_fetch_stat_tabentry() -
*
* Support function for the SQL-callable pgstat* functions. Returns
- * the collected statistics for one table or NULL. NULL doesn't mean
+ * the activity statistics for one table or NULL. NULL doesn't mean
* that the table doesn't exist, it is just not yet known by the
- * collector, so the caller is better off to report ZERO instead.
+ * activity statistics facilities, so the caller is better off to
+ * report ZERO instead.
* ----------
*/
PgStat_StatTabEntry *
pgstat_fetch_stat_tabentry(Oid relid)
{
- Oid dbid;
- PgStat_StatDBEntry *dbentry;
PgStat_StatTabEntry *tabentry;
- /*
- * If not done for this transaction, read the statistics collector stats
- * file into some hash tables.
- */
- backend_read_statsfile();
-
- /*
- * Lookup our database, then look in its table hash table.
- */
- dbid = MyDatabaseId;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
+ tabentry = pgstat_fetch_stat_tabentry_extended(false, relid);
+ if (tabentry != NULL)
+ return tabentry;
/*
* If we didn't find it, maybe it's a shared table.
*/
- dbid = InvalidOid;
- dbentry = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_FIND, NULL);
- if (dbentry != NULL && dbentry->tables != NULL)
- {
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &relid,
- HASH_FIND, NULL);
- if (tabentry)
- return tabentry;
- }
-
- return NULL;
+ tabentry = pgstat_fetch_stat_tabentry_extended(true, relid);
+ return tabentry;
+}
+
+
+/* ----------
+ * pgstat_fetch_stat_tabentry_extended() -
+ *
+ * Find table stats entry on backends in dbent. The returned entry is stored
+ * in static memory so the content is valid until the next call of the same
+ * function for the different table.
+ */
+PgStat_StatTabEntry *
+pgstat_fetch_stat_tabentry_extended(bool shared, Oid reloid)
+{
+ PgStat_StatTabEntry *shent;
+ Oid dboid = (shared ? InvalidOid : MyDatabaseId);
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for the InvalidOid */
+ Assert(reloid != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_tabent_key.databaseid == dboid &&
+ cached_tabent_key.objectid == reloid)
+ return &cached_tabent;
+
+ shent = (PgStat_StatTabEntry *)
+ get_stat_entry(PGSTAT_TYPE_TABLE, dboid, reloid, true, false, NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_tabent, shent, sizeof(PgStat_StatTabEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_tabent_key.databaseid = dboid;
+ cached_tabent_key.objectid = reloid;
+
+ return &cached_tabent;
+}
+
+
+/* ----------
+ * pgstat_copy_index_counters() -
+ *
+ * Support function for index swapping. Copy a portion of the counters of the
+ * relation to specified place.
+ * ----------
+ */
+void
+pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst)
+{
+ PgStat_StatTabEntry *tabentry;
+
+ /* No point fetching tabentry when dst is NULL */
+ if (!dst)
+ return;
+
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+
+ if (!tabentry)
+ return;
+
+ dst->t_counts.t_numscans = tabentry->numscans;
+ dst->t_counts.t_tuples_returned = tabentry->tuples_returned;
+ dst->t_counts.t_tuples_fetched = tabentry->tuples_fetched;
+ dst->t_counts.t_blocks_fetched = tabentry->blocks_fetched;
+ dst->t_counts.t_blocks_hit = tabentry->blocks_hit;
}
@@ -2680,30 +3317,46 @@ pgstat_fetch_stat_tabentry(Oid relid)
*
* Support function for the SQL-callable pgstat* functions. Returns
* the collected statistics for one function or NULL.
+ *
+ * The returned entry is stored in static memory so the content is valid until
+ * the next call of the same function for the different function id.
* ----------
*/
PgStat_StatFuncEntry *
pgstat_fetch_stat_funcentry(Oid func_id)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry = NULL;
-
- /* load the stats file if needed */
- backend_read_statsfile();
-
- /* Lookup our database, then find the requested function. */
- dbentry = pgstat_fetch_stat_dbentry(MyDatabaseId);
- if (dbentry != NULL && dbentry->functions != NULL)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &func_id,
- HASH_FIND, NULL);
- }
-
- return funcentry;
+ PgStat_StatFuncEntry *shent;
+ Oid dboid = MyDatabaseId;
+
+ /* should be called from backends */
+ Assert(IsUnderPostmaster);
+
+ /* the simple cache doesn't work properly for the InvalidOid */
+ Assert(func_id != InvalidOid);
+
+ /* Return cached result if it is valid. */
+ if (cached_funcent_key.databaseid == dboid &&
+ cached_funcent_key.objectid == func_id)
+ return &cached_funcent;
+
+ shent = (PgStat_StatFuncEntry *)
+ get_stat_entry(PGSTAT_TYPE_FUNCTION, dboid, func_id, true, false,
+ NULL);
+
+ if (!shent)
+ return NULL;
+
+ LWLockAcquire(&shent->header.lock, LW_SHARED);
+ memcpy(&cached_funcent, shent, sizeof(PgStat_StatFuncEntry));
+ LWLockRelease(&shent->header.lock);
+
+ /* remember the key for the cached entry */
+ cached_funcent_key.databaseid = dboid;
+ cached_funcent_key.objectid = func_id;
+
+ return &cached_funcent;
}
-
/* ----------
* pgstat_fetch_stat_beentry() -
*
@@ -2763,53 +3416,160 @@ pgstat_fetch_stat_numbackends(void)
return localNumBackends;
}
+/*
+ * ---------
+ * pgstat_get_stat_timestamp() -
+ *
+ * Returns the last update timstamp of global staticstics.
+ */
+TimestampTz
+pgstat_get_stat_timestamp(void)
+{
+ return (TimestampTz) pg_atomic_read_u64(&StatsShmem->stats_timestamp);
+}
+
/*
* ---------
* pgstat_fetch_stat_archiver() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the archiver statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_ArchiverStats *
+PgStat_Archiver *
pgstat_fetch_stat_archiver(void)
{
- backend_read_statsfile();
+ PgStat_Archiver reset;
+ PgStat_Archiver *reset_shared = &StatsShmem->archiver_reset_offset;
+ PgStat_Archiver *shared = &StatsShmem->archiver_stats;
+ PgStat_Archiver *cached = &cached_archiverstats;
- return &archiverStats;
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_Archiver),
+ &StatsShmem->archiver_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_Archiver));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ if (cached->archived_count == reset.archived_count)
+ {
+ cached->last_archived_wal[0] = 0;
+ cached->last_archived_timestamp = 0;
+ }
+ cached->archived_count -= reset.archived_count;
+
+ if (cached->failed_count == reset.failed_count)
+ {
+ cached->last_failed_wal[0] = 0;
+ cached->last_failed_timestamp = 0;
+ }
+ cached->failed_count -= reset.failed_count;
+
+ cached->stat_reset_timestamp = reset.stat_reset_timestamp;
+
+ cached_archiverstats_is_valid = true;
+
+ return &cached_archiverstats;
}
/*
* ---------
- * pgstat_fetch_global() -
+ * pgstat_fetch_stat_bgwriter() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the global statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_GlobalStats *
-pgstat_fetch_global(void)
+PgStat_BgWriter *
+pgstat_fetch_stat_bgwriter(void)
{
- backend_read_statsfile();
+ PgStat_BgWriter reset;
+ PgStat_BgWriter *reset_shared = &StatsShmem->bgwriter_reset_offset;
+ PgStat_BgWriter *shared = &StatsShmem->bgwriter_stats;
+ PgStat_BgWriter *cached = &cached_bgwriterstats;
+
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_BgWriter),
+ &StatsShmem->bgwriter_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_BgWriter));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ cached->buf_written_clean -= reset.buf_written_clean;
+ cached->maxwritten_clean -= reset.maxwritten_clean;
+ cached->buf_alloc -= reset.buf_alloc;
+ cached->stat_reset_timestamp = reset.stat_reset_timestamp;
+
+ cached_bgwriterstats_is_valid = true;
+
+ return &cached_bgwriterstats;
+}
+
+/*
+ * ---------
+ * pgstat_fetch_stat_checkpinter() -
+ *
+ * Support function for the SQL-callable pgstat* functions. The returned
+ * entry is stored in static memory so the content is valid until the next
+ * call.
+ * ---------
+ */
+PgStat_CheckPointer *
+pgstat_fetch_stat_checkpointer(void)
+{
+ PgStat_CheckPointer reset;
+ PgStat_CheckPointer *reset_shared = &StatsShmem->checkpointer_reset_offset;
+ PgStat_CheckPointer *shared = &StatsShmem->checkpointer_stats;
+ PgStat_CheckPointer *cached = &cached_checkpointerstats;
+
+ pgstat_copy_global_stats(cached, shared, sizeof(PgStat_CheckPointer),
+ &StatsShmem->checkpointer_changecount);
+
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&reset, reset_shared, sizeof(PgStat_CheckPointer));
+ LWLockRelease(StatsLock);
+
+ /* compensate by reset offsets */
+ cached->timed_checkpoints -= reset.timed_checkpoints;
+ cached->requested_checkpoints -= reset.requested_checkpoints;
+ cached->buf_written_checkpoints -= reset.buf_written_checkpoints;
+ cached->buf_written_backend -= reset.buf_written_backend;
+ cached->buf_fsync_backend -= reset.buf_fsync_backend;
+ cached->checkpoint_write_time -= reset.checkpoint_write_time;
+ cached->checkpoint_sync_time -= reset.checkpoint_sync_time;
+
+ cached_checkpointerstats_is_valid = true;
- return &globalStats;
+ return &cached_checkpointerstats;
}
/*
* ---------
* pgstat_fetch_stat_wal() -
*
- * Support function for the SQL-callable pgstat* functions. Returns
- * a pointer to the WAL statistics struct.
+ * Support function for the SQL-callable pgstat* functions. The returned entry
+ * is stored in static memory so the content is valid until the next
+ * call.
* ---------
*/
-PgStat_WalStats *
+PgStat_Wal *
pgstat_fetch_stat_wal(void)
{
- backend_read_statsfile();
+ if (!cached_walstats_is_valid)
+ {
+ LWLockAcquire(StatsLock, LW_SHARED);
+ memcpy(&cached_walstats, &StatsShmem->wal_stats, sizeof(PgStat_Wal));
+ LWLockRelease(StatsLock);
+ }
- return &walStats;
+ cached_walstats_is_valid = true;
+
+ return &cached_walstats;
}
/*
@@ -2823,9 +3583,27 @@ pgstat_fetch_stat_wal(void)
PgStat_SLRUStats *
pgstat_fetch_slru(void)
{
- backend_read_statsfile();
+ size_t size = sizeof(PgStat_SLRUStats) * SLRU_NUM_ELEMENTS;
- return slruStats;
+ for (;;)
+ {
+ uint32 before_count;
+ uint32 after_count;
+
+ pg_read_barrier();
+ before_count = pg_atomic_read_u32(&StatsShmem->slru_changecount);
+ memcpy(&cached_slrustats, &StatsShmem->slru_stats, size);
+ after_count = pg_atomic_read_u32(&StatsShmem->slru_changecount);
+
+ if (before_count == after_count && (before_count & 1) == 0)
+ break;
+
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ cached_slrustats_is_valid = true;
+
+ return &cached_slrustats;
}
/*
@@ -2837,13 +3615,41 @@ pgstat_fetch_slru(void)
* number of entries in nslots_p.
* ---------
*/
-PgStat_ReplSlotStats *
+PgStat_ReplSlot *
pgstat_fetch_replslot(int *nslots_p)
{
- backend_read_statsfile();
- *nslots_p = nReplSlotStats;
- return replSlotStats;
+ if (cached_replslotstats == NULL)
+ {
+ cached_replslotstats = (PgStat_ReplSlot *)
+ MemoryContextAlloc(pgStatCacheContext,
+ sizeof(PgStat_ReplSlot) * max_replication_slots);
+ }
+
+ if (n_cached_replslotstats < 0)
+ {
+ int n = 0;
+ int i;
+
+ for (i = 0 ; i < max_replication_slots ; i++)
+ {
+ PgStat_ReplSlot *shent = (PgStat_ReplSlot *)
+ get_stat_entry(PGSTAT_TYPE_REPLSLOT, MyDatabaseId, i,
+ false, false, NULL);
+ if (shent && !shent->header.dropped)
+ {
+ memcpy(cached_replslotstats[n++].slotname,
+ shent->slotname,
+ sizeof(PgStat_ReplSlot) -
+ offsetof(PgStat_ReplSlot, slotname));
+ }
+ }
+
+ n_cached_replslotstats = n;
+ }
+
+ *nslots_p = n_cached_replslotstats;
+ return cached_replslotstats;
}
/* ------------------------------------------------------------
@@ -3068,8 +3874,8 @@ pgstat_initialize(void)
*/
prevWalUsage = pgWalUsage;
- /* Set up a process-exit hook to clean up */
- on_shmem_exit(pgstat_beshutdown_hook, 0);
+ /* need to be called before dsm shutdown */
+ before_shmem_exit(pgstat_beshutdown_hook, 0);
}
/* ----------
@@ -3246,12 +4052,15 @@ pgstat_bestart(void)
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
+
+ /* attach shared database stats area */
+ attach_shared_stats();
}
/*
* Shut down a single backend's statistics reporting at process exit.
*
- * Flush any remaining statistics counts out to the collector.
+ * Flush any remaining statistics counts out to shared stats.
* Without this, operations triggered during backend exit (such as
* temp table deletions) won't be counted.
*
@@ -3264,13 +4073,22 @@ pgstat_beshutdown_hook(int code, Datum arg)
/*
* If we got as far as discovering our own database ID, we can report what
- * we did to the collector. Otherwise, we'd be sending an invalid
+ * we did to the shares stats. Otherwise, we'd be sending an invalid
* database ID, so forget it. (This means that accesses to pg_database
* during failed backend starts might never get counted.)
*/
if (OidIsValid(MyDatabaseId))
pgstat_report_stat(true);
+ /*
+ * We need to clean up temporary slots before detaching shared statistics
+ * so that the statistics for temporary slots are properly removed.
+ */
+ if (MyReplicationSlot != NULL)
+ ReplicationSlotRelease();
+
+ ReplicationSlotCleanup();
+
/*
* Clear my status entry, following the protocol of bumping st_changecount
* before and after. We use a volatile pointer here to ensure the
@@ -3281,6 +4099,8 @@ pgstat_beshutdown_hook(int code, Datum arg)
beentry->st_procpid = 0; /* mark invalid */
PGSTAT_END_WRITE_ACTIVITY(beentry);
+
+ detach_shared_stats(true);
}
@@ -3541,7 +4361,8 @@ pgstat_read_current_status(void)
#endif
int i;
- Assert(!pgStatRunningInCollector);
+ Assert(IsUnderPostmaster);
+
if (localBackendStatusTable)
return; /* already done */
@@ -3836,8 +4657,8 @@ pgstat_get_wait_activity(WaitEventActivity w)
case WAIT_EVENT_LOGICAL_LAUNCHER_MAIN:
event_name = "LogicalLauncherMain";
break;
- case WAIT_EVENT_PGSTAT_MAIN:
- event_name = "PgStatMain";
+ case WAIT_EVENT_READING_STATS_FILE:
+ event_name = "ReadingStatsFile";
break;
case WAIT_EVENT_RECOVERY_WAL_STREAM:
event_name = "RecoveryWalStream";
@@ -4491,94 +5312,80 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
/* ----------
- * pgstat_setheader() -
+ * pgstat_report_archiver() -
*
- * Set common header fields in a statistics message
+ * Report archiver statistics
* ----------
*/
-static void
-pgstat_setheader(PgStat_MsgHdr *hdr, StatMsgType mtype)
-{
- hdr->m_type = mtype;
-}
-
-
-/* ----------
- * pgstat_send() -
- *
- * Send out one statistics message to the collector
- * ----------
- */
-static void
-pgstat_send(void *msg, int len)
+void
+pgstat_report_archiver(const char *xlog, bool failed)
{
- int rc;
+ TimestampTz now = GetCurrentTimestamp();
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
- if (pgStatSock == PGINVALID_SOCKET)
- return;
- ((PgStat_MsgHdr *) msg)->m_size = len;
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->archiver_changecount, 1);
+ Assert((before_count & 1) == 0);
- /* We'll retry after EINTR, but ignore all other failures */
- do
+ if (failed)
{
- rc = send(pgStatSock, msg, len, 0);
- } while (rc < 0 && errno == EINTR);
-
-#ifdef USE_ASSERT_CHECKING
- /* In debug builds, log send failures ... */
- if (rc < 0)
- elog(LOG, "could not send to statistics collector: %m");
-#endif
-}
-
-/* ----------
- * pgstat_send_archiver() -
- *
- * Tell the collector about the WAL file that we successfully
- * archived or failed to archive.
- * ----------
- */
-void
-pgstat_send_archiver(const char *xlog, bool failed)
-{
- PgStat_MsgArchiver msg;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ARCHIVER);
- msg.m_failed = failed;
- strlcpy(msg.m_xlog, xlog, sizeof(msg.m_xlog));
- msg.m_timestamp = GetCurrentTimestamp();
- pgstat_send(&msg, sizeof(msg));
+ ++StatsShmem->archiver_stats.failed_count;
+ memcpy(&StatsShmem->archiver_stats.last_failed_wal, xlog,
+ sizeof(StatsShmem->archiver_stats.last_failed_wal));
+ StatsShmem->archiver_stats.last_failed_timestamp = now;
+ }
+ else
+ {
+ ++StatsShmem->archiver_stats.archived_count;
+ memcpy(&StatsShmem->archiver_stats.last_archived_wal, xlog,
+ sizeof(StatsShmem->archiver_stats.last_archived_wal));
+ StatsShmem->archiver_stats.last_archived_timestamp = now;
+ }
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->archiver_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
}
/* ----------
- * pgstat_send_bgwriter() -
+ * pgstat_report_bgwriter() -
*
- * Send bgwriter statistics to the collector
+ * Report bgwriter statistics
* ----------
*/
void
-pgstat_send_bgwriter(void)
+pgstat_report_bgwriter(void)
{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgBgWriter all_zeroes;
+ static const PgStat_BgWriter all_zeroes;
+ PgStat_BgWriter *s = &StatsShmem->bgwriter_stats;
+ PgStat_BgWriter *l = &BgWriterStats;
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_MsgBgWriter)) == 0)
+ if (memcmp(&BgWriterStats, &all_zeroes, sizeof(PgStat_BgWriter)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&BgWriterStats.m_hdr, PGSTAT_MTYPE_BGWRITER);
- pgstat_send(&BgWriterStats, sizeof(BgWriterStats));
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->bgwriter_changecount, 1);
+ Assert((before_count & 1) == 0);
+
+ s->buf_written_clean += l->buf_written_clean;
+ s->maxwritten_clean += l->maxwritten_clean;
+ s->buf_alloc += l->buf_alloc;
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->bgwriter_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
/*
* Clear out the statistics buffer, so it can be re-used.
@@ -4587,537 +5394,136 @@ pgstat_send_bgwriter(void)
}
/* ----------
- * pgstat_send_wal() -
+ * pgstat_report_checkpointer() -
*
- * Send WAL statistics to the collector
+ * Report checkpointer statistics
* ----------
*/
void
-pgstat_send_wal(void)
+pgstat_report_checkpointer(void)
{
/* We assume this initializes to zeroes */
- static const PgStat_MsgWal all_zeroes;
-
- WalUsage walusage;
-
- /*
- * Calculate how much WAL usage counters are increased by substracting the
- * previous counters from the current ones. Fill the results in WAL stats
- * message.
- */
- MemSet(&walusage, 0, sizeof(WalUsage));
- WalUsageAccumDiff(&walusage, &pgWalUsage, &prevWalUsage);
-
- WalStats.m_wal_records = walusage.wal_records;
- WalStats.m_wal_fpi = walusage.wal_fpi;
- WalStats.m_wal_bytes = walusage.wal_bytes;
+ static const PgStat_CheckPointer all_zeroes;
+ PgStat_CheckPointer *s = &StatsShmem->checkpointer_stats;
+ PgStat_CheckPointer *l = &CheckPointerStats;
+ uint32 before_count PG_USED_FOR_ASSERTS_ONLY;
+ uint32 after_count PG_USED_FOR_ASSERTS_ONLY;
/*
* This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
+ * this case, avoid taking lock for a completely empty stats.
*/
- if (memcmp(&WalStats, &all_zeroes, sizeof(PgStat_MsgWal)) == 0)
+ if (memcmp(&CheckPointerStats, &all_zeroes,
+ sizeof(PgStat_CheckPointer)) == 0)
return;
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&WalStats.m_hdr, PGSTAT_MTYPE_WAL);
- pgstat_send(&WalStats, sizeof(WalStats));
+ START_CRIT_SECTION();
+ before_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->checkpointer_changecount, 1);
+ Assert((before_count & 1) == 0);
- /*
- * Save the current counters for the subsequent calculation of WAL usage.
- */
- prevWalUsage = pgWalUsage;
+ s->timed_checkpoints += l->timed_checkpoints;
+ s->requested_checkpoints += l->requested_checkpoints;
+ s->checkpoint_write_time += l->checkpoint_write_time;
+ s->checkpoint_sync_time += l->checkpoint_sync_time;
+ s->buf_written_checkpoints += l->buf_written_checkpoints;
+ s->buf_written_backend += l->buf_written_backend;
+ s->buf_fsync_backend += l->buf_fsync_backend;
+
+ after_count =
+ pg_atomic_fetch_add_u32(&StatsShmem->checkpointer_changecount, 1);
+ Assert(after_count == before_count + 1);
+ END_CRIT_SECTION();
/*
* Clear out the statistics buffer, so it can be re-used.
*/
- MemSet(&WalStats, 0, sizeof(WalStats));
+ MemSet(&CheckPointerStats, 0, sizeof(CheckPointerStats));
}
/* ----------
- * pgstat_send_slru() -
+ * pgstat_report_wal() -
*
- * Send SLRU statistics to the collector
+ * Report WAL statistics
* ----------
*/
-static void
-pgstat_send_slru(void)
+void
+pgstat_report_wal(void)
{
- /* We assume this initializes to zeroes */
- static const PgStat_MsgSLRU all_zeroes;
-
- for (int i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /*
- * This function can be called even if nothing at all has happened. In
- * this case, avoid sending a completely empty message to the stats
- * collector.
- */
- if (memcmp(&SLRUStats[i], &all_zeroes, sizeof(PgStat_MsgSLRU)) == 0)
- continue;
-
- /* set the SLRU type before each send */
- SLRUStats[i].m_index = i;
-
- /*
- * Prepare and send the message
- */
- pgstat_setheader(&SLRUStats[i].m_hdr, PGSTAT_MTYPE_SLRU);
- pgstat_send(&SLRUStats[i], sizeof(PgStat_MsgSLRU));
-
- /*
- * Clear out the statistics buffer, so it can be re-used.
- */
- MemSet(&SLRUStats[i], 0, sizeof(PgStat_MsgSLRU));
- }
+ flush_walstat(false);
}
-
/* ----------
- * PgstatCollectorMain() -
+ * get_local_dbstat_entry() -
*
- * Start up the statistics collector process. This is the body of the
- * postmaster child process.
- *
- * The argc/argv parameters are valid only in EXEC_BACKEND case.
- * ----------
- */
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
-{
- int len;
- PgStat_Msg msg;
- int wr;
- WaitEvent event;
- WaitEventSet *wes;
-
- /*
- * Ignore all signals usually bound to some action in the postmaster,
- * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
- * support latch operations, because we only use a local latch.
- */
- pqsignal(SIGHUP, SignalHandlerForConfigReload);
- pqsignal(SIGINT, SIG_IGN);
- pqsignal(SIGTERM, SIG_IGN);
- pqsignal(SIGQUIT, SignalHandlerForShutdownRequest);
- pqsignal(SIGALRM, SIG_IGN);
- pqsignal(SIGPIPE, SIG_IGN);
- pqsignal(SIGUSR1, SIG_IGN);
- pqsignal(SIGUSR2, SIG_IGN);
- /* Reset some signals that are accepted by postmaster but not here */
- pqsignal(SIGCHLD, SIG_DFL);
- PG_SETMASK(&UnBlockSig);
-
- MyBackendType = B_STATS_COLLECTOR;
- init_ps_display(NULL);
-
- /*
- * Read in existing stats files or initialize the stats to zero.
- */
- pgStatRunningInCollector = true;
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
-
- /* Prepare to wait for our latch or data in our socket. */
- wes = CreateWaitEventSet(CurrentMemoryContext, 3);
- AddWaitEventToSet(wes, WL_LATCH_SET, PGINVALID_SOCKET, MyLatch, NULL);
- AddWaitEventToSet(wes, WL_POSTMASTER_DEATH, PGINVALID_SOCKET, NULL, NULL);
- AddWaitEventToSet(wes, WL_SOCKET_READABLE, pgStatSock, NULL, NULL);
-
- /*
- * Loop to process messages until we get SIGQUIT or detect ungraceful
- * death of our parent postmaster.
- *
- * For performance reasons, we don't want to do ResetLatch/WaitLatch after
- * every message; instead, do that only after a recv() fails to obtain a
- * message. (This effectively means that if backends are sending us stuff
- * like mad, we won't notice postmaster death until things slack off a
- * bit; which seems fine.) To do that, we have an inner loop that
- * iterates as long as recv() succeeds. We do check ConfigReloadPending
- * inside the inner loop, which means that such interrupts will get
- * serviced but the latch won't get cleared until next time there is a
- * break in the action.
- */
- for (;;)
- {
- /* Clear any already-pending wakeups */
- ResetLatch(MyLatch);
-
- /*
- * Quit if we get SIGQUIT from the postmaster.
- */
- if (ShutdownRequestPending)
- break;
-
- /*
- * Inner loop iterates as long as we keep getting messages, or until
- * ShutdownRequestPending becomes set.
- */
- while (!ShutdownRequestPending)
- {
- /*
- * Reload configuration if we got SIGHUP from the postmaster.
- */
- if (ConfigReloadPending)
- {
- ConfigReloadPending = false;
- ProcessConfigFile(PGC_SIGHUP);
- }
-
- /*
- * Write the stats file(s) if a new request has arrived that is
- * not satisfied by existing file(s).
- */
- if (pgstat_write_statsfile_needed())
- pgstat_write_statsfiles(false, false);
-
- /*
- * Try to receive and process a message. This will not block,
- * since the socket is set to non-blocking mode.
- *
- * XXX On Windows, we have to force pgwin32_recv to cooperate,
- * despite the previous use of pg_set_noblock() on the socket.
- * This is extremely broken and should be fixed someday.
- */
-#ifdef WIN32
- pgwin32_noblock = 1;
-#endif
-
- len = recv(pgStatSock, (char *) &msg,
- sizeof(PgStat_Msg), 0);
-
-#ifdef WIN32
- pgwin32_noblock = 0;
-#endif
-
- if (len < 0)
- {
- if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
- break; /* out of inner loop */
- ereport(ERROR,
- (errcode_for_socket_access(),
- errmsg("could not read statistics message: %m")));
- }
-
- /*
- * We ignore messages that are smaller than our common header
- */
- if (len < sizeof(PgStat_MsgHdr))
- continue;
-
- /*
- * The received length must match the length in the header
- */
- if (msg.msg_hdr.m_size != len)
- continue;
-
- /*
- * O.K. - we accept this message. Process it.
- */
- switch (msg.msg_hdr.m_type)
- {
- case PGSTAT_MTYPE_DUMMY:
- break;
-
- case PGSTAT_MTYPE_INQUIRY:
- pgstat_recv_inquiry(&msg.msg_inquiry, len);
- break;
-
- case PGSTAT_MTYPE_TABSTAT:
- pgstat_recv_tabstat(&msg.msg_tabstat, len);
- break;
-
- case PGSTAT_MTYPE_TABPURGE:
- pgstat_recv_tabpurge(&msg.msg_tabpurge, len);
- break;
-
- case PGSTAT_MTYPE_DROPDB:
- pgstat_recv_dropdb(&msg.msg_dropdb, len);
- break;
-
- case PGSTAT_MTYPE_RESETCOUNTER:
- pgstat_recv_resetcounter(&msg.msg_resetcounter, len);
- break;
-
- case PGSTAT_MTYPE_RESETSHAREDCOUNTER:
- pgstat_recv_resetsharedcounter(&msg.msg_resetsharedcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSINGLECOUNTER:
- pgstat_recv_resetsinglecounter(&msg.msg_resetsinglecounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETSLRUCOUNTER:
- pgstat_recv_resetslrucounter(&msg.msg_resetslrucounter,
- len);
- break;
-
- case PGSTAT_MTYPE_RESETREPLSLOTCOUNTER:
- pgstat_recv_resetreplslotcounter(&msg.msg_resetreplslotcounter,
- len);
- break;
-
- case PGSTAT_MTYPE_AUTOVAC_START:
- pgstat_recv_autovac(&msg.msg_autovacuum_start, len);
- break;
-
- case PGSTAT_MTYPE_VACUUM:
- pgstat_recv_vacuum(&msg.msg_vacuum, len);
- break;
-
- case PGSTAT_MTYPE_ANALYZE:
- pgstat_recv_analyze(&msg.msg_analyze, len);
- break;
-
- case PGSTAT_MTYPE_ARCHIVER:
- pgstat_recv_archiver(&msg.msg_archiver, len);
- break;
-
- case PGSTAT_MTYPE_BGWRITER:
- pgstat_recv_bgwriter(&msg.msg_bgwriter, len);
- break;
-
- case PGSTAT_MTYPE_WAL:
- pgstat_recv_wal(&msg.msg_wal, len);
- break;
-
- case PGSTAT_MTYPE_SLRU:
- pgstat_recv_slru(&msg.msg_slru, len);
- break;
-
- case PGSTAT_MTYPE_FUNCSTAT:
- pgstat_recv_funcstat(&msg.msg_funcstat, len);
- break;
-
- case PGSTAT_MTYPE_FUNCPURGE:
- pgstat_recv_funcpurge(&msg.msg_funcpurge, len);
- break;
-
- case PGSTAT_MTYPE_RECOVERYCONFLICT:
- pgstat_recv_recoveryconflict(&msg.msg_recoveryconflict,
- len);
- break;
-
- case PGSTAT_MTYPE_DEADLOCK:
- pgstat_recv_deadlock(&msg.msg_deadlock, len);
- break;
-
- case PGSTAT_MTYPE_TEMPFILE:
- pgstat_recv_tempfile(&msg.msg_tempfile, len);
- break;
-
- case PGSTAT_MTYPE_CHECKSUMFAILURE:
- pgstat_recv_checksum_failure(&msg.msg_checksumfailure,
- len);
- break;
-
- case PGSTAT_MTYPE_REPLSLOT:
- pgstat_recv_replslot(&msg.msg_replslot, len);
- break;
-
- default:
- break;
- }
- } /* end of inner message-processing loop */
-
- /* Sleep until there's something to do */
-#ifndef WIN32
- wr = WaitEventSetWait(wes, -1L, &event, 1, WAIT_EVENT_PGSTAT_MAIN);
-#else
-
- /*
- * Windows, at least in its Windows Server 2003 R2 incarnation,
- * sometimes loses FD_READ events. Waking up and retrying the recv()
- * fixes that, so don't sleep indefinitely. This is a crock of the
- * first water, but until somebody wants to debug exactly what's
- * happening there, this is the best we can do. The two-second
- * timeout matches our pre-9.2 behavior, and needs to be short enough
- * to not provoke "using stale statistics" complaints from
- * backend_read_statsfile.
- */
- wr = WaitEventSetWait(wes, 2 * 1000L /* msec */ , &event, 1,
- WAIT_EVENT_PGSTAT_MAIN);
-#endif
-
- /*
- * Emergency bailout if postmaster has died. This is to avoid the
- * necessity for manual cleanup of all postmaster children.
- */
- if (wr == 1 && event.events == WL_POSTMASTER_DEATH)
- break;
- } /* end of outer loop */
-
- /*
- * Save the final stats to reuse at next startup.
- */
- pgstat_write_statsfiles(true, true);
-
- FreeWaitEventSet(wes);
-
- exit(0);
-}
-
-/*
- * Subroutine to clear stats in a database entry
- *
- * Tables and functions hashes are initialized to empty.
- */
-static void
-reset_dbentry_counters(PgStat_StatDBEntry *dbentry)
-{
- HASHCTL hash_ctl;
-
- dbentry->n_xact_commit = 0;
- dbentry->n_xact_rollback = 0;
- dbentry->n_blocks_fetched = 0;
- dbentry->n_blocks_hit = 0;
- dbentry->n_tuples_returned = 0;
- dbentry->n_tuples_fetched = 0;
- dbentry->n_tuples_inserted = 0;
- dbentry->n_tuples_updated = 0;
- dbentry->n_tuples_deleted = 0;
- dbentry->last_autovac_time = 0;
- dbentry->n_conflict_tablespace = 0;
- dbentry->n_conflict_lock = 0;
- dbentry->n_conflict_snapshot = 0;
- dbentry->n_conflict_bufferpin = 0;
- dbentry->n_conflict_startup_deadlock = 0;
- dbentry->n_temp_files = 0;
- dbentry->n_temp_bytes = 0;
- dbentry->n_deadlocks = 0;
- dbentry->n_checksum_failures = 0;
- dbentry->last_checksum_failure = 0;
- dbentry->n_block_read_time = 0;
- dbentry->n_block_write_time = 0;
-
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
- dbentry->stats_timestamp = 0;
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS);
-}
-
-/*
- * Lookup the hash table entry for the specified database. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+ * Find or create a local PgStat_StatDBEntry entry for dbid. New entry is
+ * created and initialized if not exists.
*/
static PgStat_StatDBEntry *
-pgstat_get_db_entry(Oid databaseid, bool create)
+get_local_dbstat_entry(Oid dbid)
{
- PgStat_StatDBEntry *result;
+ PgStat_StatDBEntry *dbentry;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
-
- /* Lookup or create the hash table entry for this database */
- result = (PgStat_StatDBEntry *) hash_search(pgStatDBHash,
- &databaseid,
- action, &found);
-
- if (!create && !found)
- return NULL;
/*
- * If not found, initialize the new one. This creates empty hash tables
- * for tables and functions, too.
+ * Find an entry or create a new one.
*/
- if (!found)
- reset_dbentry_counters(result);
+ dbentry = (PgStat_StatDBEntry *)
+ get_local_stat_entry(PGSTAT_TYPE_DB, dbid, InvalidOid,
+ true, &found);
- return result;
+ return dbentry;
}
-
-/*
- * Lookup the hash table entry for the specified table. If no hash
- * table entry exists, initialize it, if the create parameter is true.
- * Else, return NULL.
+/* ----------
+ * get_local_tabstat_entry() -
+ * Find or create a PgStat_TableStatus entry for rel. New entry is created and
+ * initialized if not exists.
+ * ----------
*/
-static PgStat_StatTabEntry *
-pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
+static PgStat_TableStatus *
+get_local_tabstat_entry(Oid rel_id, bool isshared)
{
- PgStat_StatTabEntry *result;
+ PgStat_TableStatus *tabentry;
bool found;
- HASHACTION action = (create ? HASH_ENTER : HASH_FIND);
- /* Lookup or create the hash table entry for this table */
- result = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- &tableoid,
- action, &found);
+ tabentry = (PgStat_TableStatus *)
+ get_local_stat_entry(PGSTAT_TYPE_TABLE,
+ isshared ? InvalidOid : MyDatabaseId,
+ rel_id, true, &found);
- if (!create && !found)
- return NULL;
-
- /* If not found, initialize the new one. */
- if (!found)
- {
- result->numscans = 0;
- result->tuples_returned = 0;
- result->tuples_fetched = 0;
- result->tuples_inserted = 0;
- result->tuples_updated = 0;
- result->tuples_deleted = 0;
- result->tuples_hot_updated = 0;
- result->n_live_tuples = 0;
- result->n_dead_tuples = 0;
- result->changes_since_analyze = 0;
- result->inserts_since_vacuum = 0;
- result->blocks_fetched = 0;
- result->blocks_hit = 0;
- result->vacuum_timestamp = 0;
- result->vacuum_count = 0;
- result->autovac_vacuum_timestamp = 0;
- result->autovac_vacuum_count = 0;
- result->analyze_timestamp = 0;
- result->analyze_count = 0;
- result->autovac_analyze_timestamp = 0;
- result->autovac_analyze_count = 0;
- }
-
- return result;
+ return tabentry;
}
-
/* ----------
- * pgstat_write_statsfiles() -
- * Write the global statistics file, as well as requested DB files.
+ * pgstat_write_statsfile() -
+ * Write the global statistics file, as well as DB files.
*
- * 'permanent' specifies writing to the permanent files not temporary ones.
- * When true (happens only when the collector is shutting down), also remove
- * the temporary files so that backends starting up under a new postmaster
- * can't read old data before the new collector is ready.
- *
- * When 'allDbs' is false, only the requested databases (listed in
- * pending_write_requests) will be written; otherwise, all databases
- * will be written.
+ * This function is called in the last process that is accessing the shared
+ * stats so locking is not required.
* ----------
*/
static void
-pgstat_write_statsfiles(bool permanent, bool allDbs)
+pgstat_write_statsfile(void)
{
- HASH_SEQ_STATUS hstat;
- PgStat_StatDBEntry *dbentry;
FILE *fpout;
int32 format_id;
- const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ const char *tmpfile = PGSTAT_STAT_PERMANENT_TMPFILE;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
int rc;
- int i;
+ dshash_seq_status hstat;
+ PgStatHashEntry *ps;
+
+ /* stats is not initialized yet. just return. */
+ if (StatsShmem->stats_dsa_handle == DSM_HANDLE_INVALID)
+ return;
+
+ /* this is the last process that is accesing the shared stats */
+#ifdef USE_ASSERT_CHECKING
+ LWLockAcquire(StatsLock, LW_SHARED);
+ Assert(StatsShmem->refcount == 0);
+ LWLockRelease(StatsLock);
+#endif
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -5137,7 +5543,7 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
/*
* Set the timestamp of the stats file.
*/
- globalStats.stats_timestamp = GetCurrentTimestamp();
+ pg_atomic_write_u64(&StatsShmem->stats_timestamp, GetCurrentTimestamp());
/*
* Write the file header --- currently just a format ID.
@@ -5147,200 +5553,72 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
(void) rc; /* we'll check for error with ferror */
/*
- * Write global stats struct
+ * Write bgwriter global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
+ rc = fwrite(&StatsShmem->bgwriter_stats, sizeof(PgStat_BgWriter), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Write archiver stats struct
+ * Write checkpointer global stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
+ rc = fwrite(&StatsShmem->checkpointer_stats, sizeof(PgStat_CheckPointer), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Write WAL stats struct
+ * Write archiver global stats struct
*/
- rc = fwrite(&walStats, sizeof(walStats), 1, fpout);
+ rc = fwrite(&StatsShmem->archiver_stats, sizeof(PgStat_Archiver), 1,
+ fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /*
+ * Write WAL global stats struct
+ */
+ rc = fwrite(&StatsShmem->wal_stats, sizeof(PgStat_Wal), 1, fpout);
(void) rc; /* we'll check for error with ferror */
/*
* Write SLRU stats struct
*/
- rc = fwrite(slruStats, sizeof(slruStats), 1, fpout);
+ rc = fwrite(&StatsShmem->slru_stats, sizeof(PgStatSharedSLRUStats), 1,
+ fpout);
(void) rc; /* we'll check for error with ferror */
/*
- * Walk through the database table.
+ * Walk through the stats entry
*/
- hash_seq_init(&hstat, pgStatDBHash);
- while ((dbentry = (PgStat_StatDBEntry *) hash_seq_search(&hstat)) != NULL)
+ dshash_seq_init(&hstat, pgStatSharedHash, false);
+ while ((ps = dshash_seq_next(&hstat)) != NULL)
{
- /*
- * Write out the table and function stats for this DB into the
- * appropriate per-DB stat file, if required.
- */
- if (allDbs || pgstat_db_requested(dbentry->databaseid))
+ PgStat_StatEntryHeader *shent;
+ size_t len;
+
+ CHECK_FOR_INTERRUPTS();
+
+ shent = (PgStat_StatEntryHeader *)dsa_get_address(area, ps->body);
+
+ /* we may have some "dropped" entries not yet removed, skip them */
+ if (shent->dropped)
+ continue;
+
+ /* Make DB's timestamp consistent with the global stats */
+ if (ps->key.type == PGSTAT_TYPE_DB)
{
- /* Make DB's timestamp consistent with the global stats */
- dbentry->stats_timestamp = globalStats.stats_timestamp;
+ PgStat_StatDBEntry *dbentry = (PgStat_StatDBEntry *) shent;
- pgstat_write_db_statsfile(dbentry, permanent);
+ dbentry->stats_timestamp =
+ (TimestampTz) pg_atomic_read_u64(&StatsShmem->stats_timestamp);
}
- /*
- * Write out the DB entry. We don't write the tables or functions
- * pointers, since they're of no use to any other process.
- */
- fputc('D', fpout);
- rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * Write replication slot stats struct
- */
- for (i = 0; i < nReplSlotStats; i++)
- {
- fputc('R', fpout);
- rc = fwrite(&replSlotStats[i], sizeof(PgStat_ReplSlotStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
-
- /*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
- */
- fputc('E', fpout);
-
- if (ferror(fpout))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write temporary statistics file \"%s\": %m",
- tmpfile)));
- FreeFile(fpout);
- unlink(tmpfile);
- }
- else if (FreeFile(fpout) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not close temporary statistics file \"%s\": %m",
- tmpfile)));
- unlink(tmpfile);
- }
- else if (rename(tmpfile, statfile) < 0)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not rename temporary statistics file \"%s\" to \"%s\": %m",
- tmpfile, statfile)));
- unlink(tmpfile);
- }
-
- if (permanent)
- unlink(pgstat_stat_filename);
-
- /*
- * Now throw away the list of requests. Note that requests sent after we
- * started the write are still waiting on the network socket.
- */
- list_free(pending_write_requests);
- pending_write_requests = NIL;
-}
-
-/*
- * return the filename for a DB stat file; filename is the output buffer,
- * of length len.
- */
-static void
-get_dbstat_filename(bool permanent, bool tempname, Oid databaseid,
- char *filename, int len)
-{
- int printed;
-
- /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
- printed = snprintf(filename, len, "%s/db_%u.%s",
- permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
- pgstat_stat_directory,
- databaseid,
- tempname ? "tmp" : "stat");
- if (printed >= len)
- elog(ERROR, "overlength pgstat path");
-}
-
-/* ----------
- * pgstat_write_db_statsfile() -
- * Write the stat file for a single database.
- *
- * If writing to the permanent file (happens when the collector is
- * shutting down only), remove the temporary file so that backends
- * starting up under a new postmaster can't read the old data before
- * the new collector is ready.
- * ----------
- */
-static void
-pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
-{
- HASH_SEQ_STATUS tstat;
- HASH_SEQ_STATUS fstat;
- PgStat_StatTabEntry *tabentry;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpout;
- int32 format_id;
- Oid dbid = dbentry->databaseid;
- int rc;
- char tmpfile[MAXPGPATH];
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
- get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "writing stats file \"%s\"", statfile);
-
- /*
- * Open the statistics temp file to write out the current values.
- */
- fpout = AllocateFile(tmpfile, PG_BINARY_W);
- if (fpout == NULL)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not open temporary statistics file \"%s\": %m",
- tmpfile)));
- return;
- }
-
- /*
- * Write the file header --- currently just a format ID.
- */
- format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
-
- /*
- * Walk through the database's access stats per table.
- */
- hash_seq_init(&tstat, dbentry->tables);
- while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
- {
- fputc('T', fpout);
- rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
- }
+ fputc('S', fpout);
+ rc = fwrite(&ps->key, sizeof(PgStatHashKey), 1, fpout);
- /*
- * Walk through the database's function stats table.
- */
- hash_seq_init(&fstat, dbentry->functions);
- while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
- {
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
+ /* Write except the header part of the etnry */
+ len = PGSTAT_SHENT_BODY_LEN(ps->key.type);
+ rc = fwrite(PGSTAT_SHENT_BODY(shent), len, 1, fpout);
(void) rc; /* we'll check for error with ferror */
}
+ dshash_seq_term(&hstat);
/*
* No more output to be done. Close the temp file and replace the old
@@ -5374,113 +5652,63 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
tmpfile, statfile)));
unlink(tmpfile);
}
-
- if (permanent)
- {
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing temporary stats file \"%s\"", statfile);
- unlink(statfile);
- }
}
/* ----------
- * pgstat_read_statsfiles() -
- *
- * Reads in some existing statistics collector files and returns the
- * databases hash table that is the top level of the data.
+ * pgstat_read_statsfile() -
*
- * If 'onlydb' is not InvalidOid, it means we only want data for that DB
- * plus the shared catalogs ("DB 0"). We'll still populate the DB hash
- * table for all databases, but we don't bother even creating table/function
- * hash tables for other databases.
+ * Reads in existing activity statistics file into the shared stats hash.
*
- * 'permanent' specifies reading from the permanent files not temporary ones.
- * When true (happens only when the collector is starting up), remove the
- * files after reading; the in-memory status is now authoritative, and the
- * files would be out of date in case somebody else reads them.
- *
- * If a 'deep' read is requested, table/function stats are read, otherwise
- * the table/function hash tables remain empty.
+ * This function is called in the only process that is accessing the shared
+ * stats so locking is not required.
* ----------
*/
-static HTAB *
-pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
+static void
+pgstat_read_statsfile(void)
{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatDBEntry dbbuf;
- HASHCTL hash_ctl;
- HTAB *dbhash;
FILE *fpin;
int32 format_id;
bool found;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
- int i;
+ const char *statfile = PGSTAT_STAT_PERMANENT_FILENAME;
+ char tag;
- /*
- * The tables will live in pgStatLocalContext.
- */
- pgstat_setup_memcxt();
+ /* shouldn't be called from postmaster */
+ Assert(IsUnderPostmaster);
- /*
- * Create the DB hashtable
- */
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatDBEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbhash = hash_create("Databases hash", PGSTAT_DB_HASH_SIZE, &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+ /* this is the only process that is accesing the shared stats */
+#ifdef USE_ASSERT_CHECKING
+ LWLockAcquire(StatsLock, LW_SHARED);
+ Assert(StatsShmem->refcount == 1);
+ LWLockRelease(StatsLock);
+#endif
- /* Allocate the space for replication slot statistics */
- replSlotStats = palloc0(max_replication_slots * sizeof(PgStat_ReplSlotStats));
- nReplSlotStats = 0;
-
- /*
- * Clear out global, archiver, WAL and SLRU statistics so they start from
- * zero in case we can't load an existing statsfile.
- */
- memset(&globalStats, 0, sizeof(globalStats));
- memset(&archiverStats, 0, sizeof(archiverStats));
- memset(&walStats, 0, sizeof(walStats));
- memset(&slruStats, 0, sizeof(slruStats));
+ elog(DEBUG2, "reading stats file \"%s\"", statfile);
/*
* Set the current timestamp (will be kept only in case we can't load an
* existing statsfile).
*/
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- archiverStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
- walStats.stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all SLRU items too.
- */
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- slruStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
-
- /*
- * Set the same reset timestamp for all replication slots too.
- */
- for (i = 0; i < max_replication_slots; i++)
- replSlotStats[i].stat_reset_timestamp = globalStats.stat_reset_timestamp;
+ StatsShmem->bgwriter_stats.stat_reset_timestamp = GetCurrentTimestamp();
+ StatsShmem->archiver_stats.stat_reset_timestamp =
+ StatsShmem->bgwriter_stats.stat_reset_timestamp;
/*
* Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
+ * returns zero for anything and the activity statistics simply starts
+ * from scratch with empty counters.
*
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
+ * ENOENT is a possibility if the activity statistics is not running or
+ * has not yet written the stats file the first time. Any other failure
* condition is suspicious.
*/
if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
{
if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errcode_for_file_access(),
errmsg("could not open statistics file \"%s\": %m",
statfile)));
- return dbhash;
+ return;
}
/*
@@ -5489,681 +5717,150 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- goto done;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&globalStats, 0, sizeof(globalStats));
- goto done;
- }
-
- /*
- * In the collector, disregard the timestamp we read from the permanent
- * stats file; we should be willing to write a temp stats file immediately
- * upon the first request from any backend. This only matters if the old
- * file's timestamp is less than PGSTAT_STAT_INTERVAL ago, but that's not
- * an unusual scenario.
- */
- if (pgStatRunningInCollector)
- globalStats.stats_timestamp = 0;
-
- /*
- * Read archiver stats struct
- */
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&archiverStats, 0, sizeof(archiverStats));
- goto done;
- }
-
- /*
- * Read WAL stats struct
- */
- if (fread(&walStats, 1, sizeof(walStats), fpin) != sizeof(walStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&walStats, 0, sizeof(walStats));
goto done;
}
/*
- * Read SLRU stats struct
+ * Read bgwiter stats struct
*/
- if (fread(slruStats, 1, sizeof(slruStats), fpin) != sizeof(slruStats))
+ if (fread(&StatsShmem->bgwriter_stats, 1, sizeof(PgStat_BgWriter), fpin) !=
+ sizeof(PgStat_BgWriter))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- memset(&slruStats, 0, sizeof(slruStats));
+ MemSet(&StatsShmem->bgwriter_stats, 0, sizeof(PgStat_BgWriter));
goto done;
}
/*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Add to the DB hash
- */
- dbentry = (PgStat_StatDBEntry *) hash_search(dbhash,
- (void *) &dbbuf.databaseid,
- HASH_ENTER,
- &found);
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(dbentry, &dbbuf, sizeof(PgStat_StatDBEntry));
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * In the collector, disregard the timestamp we read from the
- * permanent stats file; we should be willing to write a temp
- * stats file immediately upon the first request from any
- * backend.
- */
- if (pgStatRunningInCollector)
- dbentry->stats_timestamp = 0;
-
- /*
- * Don't create tables/functions hashtables for uninteresting
- * databases.
- */
- if (onlydb != InvalidOid)
- {
- if (dbbuf.databaseid != onlydb &&
- dbbuf.databaseid != InvalidOid)
- break;
- }
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatTabEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->tables = hash_create("Per-database table",
- PGSTAT_TAB_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- hash_ctl.keysize = sizeof(Oid);
- hash_ctl.entrysize = sizeof(PgStat_StatFuncEntry);
- hash_ctl.hcxt = pgStatLocalContext;
- dbentry->functions = hash_create("Per-database function",
- PGSTAT_FUNCTION_HASH_SIZE,
- &hash_ctl,
- HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-
- /*
- * If requested, read the data from the database-specific
- * file. Otherwise we just leave the hashtables empty.
- */
- if (deep)
- pgstat_read_db_statsfile(dbentry->databaseid,
- dbentry->tables,
- dbentry->functions,
- permanent);
-
- break;
-
- /*
- * 'R' A PgStat_ReplSlotStats struct describing a replication
- * slot follows.
- */
- case 'R':
- if (fread(&replSlotStats[nReplSlotStats], 1, sizeof(PgStat_ReplSlotStats), fpin)
- != sizeof(PgStat_ReplSlotStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- memset(&replSlotStats[nReplSlotStats], 0, sizeof(PgStat_ReplSlotStats));
- goto done;
- }
- nReplSlotStats++;
- break;
-
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- /* If requested to read the permanent file, also get rid of it. */
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-
- return dbhash;
-}
-
-
-/* ----------
- * pgstat_read_db_statsfile() -
- *
- * Reads in the existing statistics collector file for the given database,
- * filling the passed-in tables and functions hash tables.
- *
- * As in pgstat_read_statsfiles, if the permanent file is requested, it is
- * removed after reading.
- *
- * Note: this code has the ability to skip storing per-table or per-function
- * data, if NULL is passed for the corresponding hashtable. That's not used
- * at the moment though.
- * ----------
- */
-static void
-pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
- bool permanent)
-{
- PgStat_StatTabEntry *tabentry;
- PgStat_StatTabEntry tabbuf;
- PgStat_StatFuncEntry funcbuf;
- PgStat_StatFuncEntry *funcentry;
- FILE *fpin;
- int32 format_id;
- bool found;
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
-
- /*
- * Try to open the stats file. If it doesn't exist, the backends simply
- * return zero for anything and the collector simply starts from scratch
- * with empty counters.
- *
- * ENOENT is a possibility if the stats collector is not running or has
- * not yet written the stats file the first time. Any other failure
- * condition is suspicious.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return;
- }
-
- /*
- * Verify it's of the expected format.
+ * Read checkpointer stats struct
*/
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
+ if (fread(&StatsShmem->checkpointer_stats, 1, sizeof(PgStat_CheckPointer), fpin) !=
+ sizeof(PgStat_CheckPointer))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
+ MemSet(&StatsShmem->checkpointer_stats, 0, sizeof(PgStat_CheckPointer));
goto done;
}
- /*
- * We found an existing collector stats file. Read it and put all the
- * hashtable entries into place.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'T' A PgStat_StatTabEntry follows.
- */
- case 'T':
- if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
- fpin) != sizeof(PgStat_StatTabEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if table data not wanted.
- */
- if (tabhash == NULL)
- break;
-
- tabentry = (PgStat_StatTabEntry *) hash_search(tabhash,
- (void *) &tabbuf.tableid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(tabentry, &tabbuf, sizeof(tabbuf));
- break;
-
- /*
- * 'F' A PgStat_StatFuncEntry follows.
- */
- case 'F':
- if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
- fpin) != sizeof(PgStat_StatFuncEntry))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- /*
- * Skip if function data not wanted.
- */
- if (funchash == NULL)
- break;
-
- funcentry = (PgStat_StatFuncEntry *) hash_search(funchash,
- (void *) &funcbuf.functionid,
- HASH_ENTER, &found);
-
- if (found)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
-
- memcpy(funcentry, &funcbuf, sizeof(funcbuf));
- break;
-
- /*
- * 'E' The EOF marker of a complete stats file.
- */
- case 'E':
- goto done;
-
- default:
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- goto done;
- }
- }
-
-done:
- FreeFile(fpin);
-
- if (permanent)
- {
- elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
- unlink(statfile);
- }
-}
-
-/* ----------
- * pgstat_read_db_statsfile_timestamp() -
- *
- * Attempt to determine the timestamp of the last db statfile write.
- * Returns true if successful; the timestamp is stored in *ts. The caller must
- * rely on timestamp stored in *ts iff the function returns true.
- *
- * This needs to be careful about handling databases for which no stats file
- * exists, such as databases without a stat entry or those not yet written:
- *
- * - if there's a database entry in the global file, return the corresponding
- * stats_timestamp value.
- *
- * - if there's no db stat entry (e.g. for a new or inactive database),
- * there's no stats_timestamp value, but also nothing to write so we return
- * the timestamp of the global statfile.
- * ----------
- */
-static bool
-pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
- TimestampTz *ts)
-{
- PgStat_StatDBEntry dbentry;
- PgStat_GlobalStats myGlobalStats;
- PgStat_ArchiverStats myArchiverStats;
- PgStat_WalStats myWalStats;
- PgStat_SLRUStats mySLRUStats[SLRU_NUM_ELEMENTS];
- PgStat_ReplSlotStats myReplSlotStats;
- FILE *fpin;
- int32 format_id;
- const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
-
- /*
- * Try to open the stats file. As above, anything but ENOENT is worthy of
- * complaining about.
- */
- if ((fpin = AllocateFile(statfile, PG_BINARY_R)) == NULL)
- {
- if (errno != ENOENT)
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errcode_for_file_access(),
- errmsg("could not open statistics file \"%s\": %m",
- statfile)));
- return false;
- }
-
- /*
- * Verify it's of the expected format.
- */
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
- format_id != PGSTAT_FILE_FORMAT_ID)
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * Read global stats struct
- */
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
/*
* Read archiver stats struct
*/
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
+ if (fread(&StatsShmem->archiver_stats, 1, sizeof(PgStat_Archiver),
+ fpin) != sizeof(PgStat_Archiver))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
+ MemSet(&StatsShmem->archiver_stats, 0, sizeof(PgStat_Archiver));
+ goto done;
}
/*
* Read WAL stats struct
*/
- if (fread(&myWalStats, 1, sizeof(myWalStats), fpin) != sizeof(myWalStats))
+ if (fread(&StatsShmem->wal_stats, 1, sizeof(PgStat_Wal), fpin)
+ != sizeof(PgStat_Wal))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
+ MemSet(&StatsShmem->wal_stats, 0, sizeof(PgStat_Wal));
+ goto done;
}
/*
* Read SLRU stats struct
*/
- if (fread(mySLRUStats, 1, sizeof(mySLRUStats), fpin) != sizeof(mySLRUStats))
+ if (fread(&StatsShmem->slru_stats, 1, sizeof(PgStatSharedSLRUStats),
+ fpin) != sizeof(PgStatSharedSLRUStats))
{
- ereport(pgStatRunningInCollector ? LOG : WARNING,
+ ereport(LOG,
(errmsg("corrupted statistics file \"%s\"", statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /* By default, we're going to return the timestamp of the global file. */
- *ts = myGlobalStats.stats_timestamp;
-
- /*
- * We found an existing collector stats file. Read it and look for a
- * record for the requested database. If found, use its timestamp.
- */
- for (;;)
- {
- switch (fgetc(fpin))
- {
- /*
- * 'D' A PgStat_StatDBEntry struct describing a database
- * follows.
- */
- case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
-
- /*
- * If this is the DB we're looking for, save its timestamp and
- * we're done.
- */
- if (dbentry.databaseid == databaseid)
- {
- *ts = dbentry.stats_timestamp;
- goto done;
- }
-
- break;
-
- /*
- * 'R' A PgStat_ReplSlotStats struct describing a replication
- * slot follows.
- */
- case 'R':
- if (fread(&myReplSlotStats, 1, sizeof(PgStat_ReplSlotStats), fpin)
- != sizeof(PgStat_ReplSlotStats))
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
- break;
-
- case 'E':
- goto done;
-
- default:
- {
- ereport(pgStatRunningInCollector ? LOG : WARNING,
- (errmsg("corrupted statistics file \"%s\"",
- statfile)));
- FreeFile(fpin);
- return false;
- }
- }
+ goto done;
}
-done:
- FreeFile(fpin);
- return true;
-}
-
-/*
- * If not already done, read the statistics collector stats file into
- * some hash tables. The results will be kept until pgstat_clear_snapshot()
- * is called (typically, at end of transaction).
- */
-static void
-backend_read_statsfile(void)
-{
- TimestampTz min_ts = 0;
- TimestampTz ref_ts = 0;
- Oid inquiry_db;
- int count;
-
- /* already read it? */
- if (pgStatDBHash)
- return;
- Assert(!pgStatRunningInCollector);
-
- /*
- * In a normal backend, we check staleness of the data for our own DB, and
- * so we send MyDatabaseId in inquiry messages. In the autovac launcher,
- * check staleness of the shared-catalog data, and send InvalidOid in
- * inquiry messages so as not to force writing unnecessary data.
- */
- if (IsAutoVacuumLauncherProcess())
- inquiry_db = InvalidOid;
- else
- inquiry_db = MyDatabaseId;
-
/*
- * Loop until fresh enough stats file is available or we ran out of time.
- * The stats inquiry message is sent repeatedly in case collector drops
- * it; but not every single time, as that just swamps the collector.
+ * We found an existing activity statistics file. Read it and put all the
+ * hash table entries into place.
*/
- for (count = 0; count < PGSTAT_POLL_LOOP_COUNT; count++)
+ while ((tag = fgetc(fpin)) == 'S')
{
- bool ok;
- TimestampTz file_ts = 0;
- TimestampTz cur_ts;
+ PgStatHashKey key;
+ PgStat_StatEntryHeader *p;
+ size_t len;
CHECK_FOR_INTERRUPTS();
- ok = pgstat_read_db_statsfile_timestamp(inquiry_db, false, &file_ts);
-
- cur_ts = GetCurrentTimestamp();
- /* Calculate min acceptable timestamp, if we didn't already */
- if (count == 0 || cur_ts < ref_ts)
+ if (fread(&key, 1, sizeof(key), fpin) != sizeof(key))
{
- /*
- * We set the minimum acceptable timestamp to PGSTAT_STAT_INTERVAL
- * msec before now. This indirectly ensures that the collector
- * needn't write the file more often than PGSTAT_STAT_INTERVAL. In
- * an autovacuum worker, however, we want a lower delay to avoid
- * using stale data, so we use PGSTAT_RETRY_DELAY (since the
- * number of workers is low, this shouldn't be a problem).
- *
- * We don't recompute min_ts after sleeping, except in the
- * unlikely case that cur_ts went backwards. So we might end up
- * accepting a file a bit older than PGSTAT_STAT_INTERVAL. In
- * practice that shouldn't happen, though, as long as the sleep
- * time is less than PGSTAT_STAT_INTERVAL; and we don't want to
- * tell the collector that our cutoff time is less than what we'd
- * actually accept.
- */
- ref_ts = cur_ts;
- if (IsAutoVacuumWorkerProcess())
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_RETRY_DELAY);
- else
- min_ts = TimestampTzPlusMilliseconds(ref_ts,
- -PGSTAT_STAT_INTERVAL);
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
}
- /*
- * If the file timestamp is actually newer than cur_ts, we must have
- * had a clock glitch (system time went backwards) or there is clock
- * skew between our processor and the stats collector's processor.
- * Accept the file, but send an inquiry message anyway to make
- * pgstat_recv_inquiry do a sanity check on the collector's time.
- */
- if (ok && file_ts > cur_ts)
- {
- /*
- * A small amount of clock skew between processors isn't terribly
- * surprising, but a large difference is worth logging. We
- * arbitrarily define "large" as 1000 msec.
- */
- if (file_ts >= TimestampTzPlusMilliseconds(cur_ts, 1000))
- {
- char *filetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- filetime = pstrdup(timestamptz_to_str(file_ts));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- ereport(LOG,
- (errmsg("statistics collector's time %s is later than backend local time %s",
- filetime, mytime)));
- pfree(filetime);
- pfree(mytime);
- }
+ p = get_stat_entry(key.type, key.databaseid, key.objectid,
+ false, true, &found);
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
- break;
+ /* don't allow duplicate entries */
+ if (found)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
}
- /* Normal acceptance case: file is not older than cutoff time */
- if (ok && file_ts >= min_ts)
- break;
+ /* Avoid overwriting header part */
+ len = PGSTAT_SHENT_BODY_LEN(key.type);
- /* Not there or too old, so kick the collector and wait a bit */
- if ((count % PGSTAT_INQ_LOOP_COUNT) == 0)
- pgstat_send_inquiry(cur_ts, min_ts, inquiry_db);
-
- pg_usleep(PGSTAT_RETRY_DELAY * 1000L);
+ if (fread(PGSTAT_SHENT_BODY(p), 1, len, fpin) != len)
+ {
+ ereport(LOG,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
}
- if (count >= PGSTAT_POLL_LOOP_COUNT)
+ if (tag != 'E')
+ {
ereport(LOG,
- (errmsg("using stale statistics instead of current ones "
- "because stats collector is not responding")));
+ (errmsg("corrupted statistics file \"%s\"",
+ statfile)));
+ goto done;
+ }
- /*
- * Autovacuum launcher wants stats about all databases, but a shallow read
- * is sufficient. Regular backends want a deep read for just the tables
- * they can see (MyDatabaseId + shared catalogs).
- */
- if (IsAutoVacuumLauncherProcess())
- pgStatDBHash = pgstat_read_statsfiles(InvalidOid, false, false);
- else
- pgStatDBHash = pgstat_read_statsfiles(MyDatabaseId, false, true);
+done:
+ FreeFile(fpin);
+
+ elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
+ unlink(statfile);
+
+ return;
}
-
/* ----------
* pgstat_setup_memcxt() -
*
- * Create pgStatLocalContext, if not already done.
+ * Create pgStatLocalContext if not already done.
* ----------
*/
static void
pgstat_setup_memcxt(void)
{
if (!pgStatLocalContext)
- pgStatLocalContext = AllocSetContextCreate(TopMemoryContext,
- "Statistics snapshot",
- ALLOCSET_SMALL_SIZES);
-}
+ pgStatLocalContext =
+ AllocSetContextCreate(TopMemoryContext,
+ "Backend statistics snapshot",
+ ALLOCSET_SMALL_SIZES);
+ if (!pgStatCacheContext)
+ pgStatCacheContext =
+ AllocSetContextCreate(CacheMemoryContext,
+ "Activity statistics",
+ ALLOCSET_SMALL_SIZES);
+}
/* ----------
* pgstat_clear_snapshot() -
@@ -6180,906 +5877,25 @@ pgstat_clear_snapshot(void)
{
/* Release memory, if any was allocated */
if (pgStatLocalContext)
+ {
MemoryContextDelete(pgStatLocalContext);
- /* Reset variables */
- pgStatLocalContext = NULL;
- pgStatDBHash = NULL;
- localBackendStatusTable = NULL;
- localNumBackends = 0;
-}
-
-
-/* ----------
- * pgstat_recv_inquiry() -
- *
- * Process stat inquiry requests.
- * ----------
- */
-static void
-pgstat_recv_inquiry(PgStat_MsgInquiry *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- elog(DEBUG2, "received inquiry for database %u", msg->databaseid);
-
- /*
- * If there's already a write request for this DB, there's nothing to do.
- *
- * Note that if a request is found, we return early and skip the below
- * check for clock skew. This is okay, since the only way for a DB
- * request to be present in the list is that we have been here since the
- * last write round. It seems sufficient to check for clock skew once per
- * write round.
- */
- if (list_member_oid(pending_write_requests, msg->databaseid))
- return;
-
- /*
- * Check to see if we last wrote this database at a time >= the requested
- * cutoff time. If so, this is a stale request that was generated before
- * we updated the DB file, and we don't need to do so again.
- *
- * If the requestor's local clock time is older than stats_timestamp, we
- * should suspect a clock glitch, ie system time going backwards; though
- * the more likely explanation is just delayed message receipt. It is
- * worth expending a GetCurrentTimestamp call to be sure, since a large
- * retreat in the system clock reading could otherwise cause us to neglect
- * to update the stats file for a long time.
- */
- dbentry = pgstat_get_db_entry(msg->databaseid, false);
- if (dbentry == NULL)
- {
- /*
- * We have no data for this DB. Enter a write request anyway so that
- * the global stats will get updated. This is needed to prevent
- * backend_read_statsfile from waiting for data that we cannot supply,
- * in the case of a new DB that nobody has yet reported any stats for.
- * See the behavior of pgstat_read_db_statsfile_timestamp.
- */
- }
- else if (msg->clock_time < dbentry->stats_timestamp)
- {
- TimestampTz cur_ts = GetCurrentTimestamp();
-
- if (cur_ts < dbentry->stats_timestamp)
- {
- /*
- * Sure enough, time went backwards. Force a new stats file write
- * to get back in sync; but first, log a complaint.
- */
- char *writetime;
- char *mytime;
-
- /* Copy because timestamptz_to_str returns a static buffer */
- writetime = pstrdup(timestamptz_to_str(dbentry->stats_timestamp));
- mytime = pstrdup(timestamptz_to_str(cur_ts));
- ereport(LOG,
- (errmsg("stats_timestamp %s is later than collector's time %s for database %u",
- writetime, mytime, dbentry->databaseid)));
- pfree(writetime);
- pfree(mytime);
- }
- else
- {
- /*
- * Nope, it's just an old request. Assuming msg's clock_time is
- * >= its cutoff_time, it must be stale, so we can ignore it.
- */
- return;
- }
- }
- else if (msg->cutoff_time <= dbentry->stats_timestamp)
- {
- /* Stale request, ignore it */
- return;
- }
-
- /*
- * We need to write this DB, so create a request.
- */
- pending_write_requests = lappend_oid(pending_write_requests,
- msg->databaseid);
-}
-
-
-/* ----------
- * pgstat_recv_tabstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_tabstat(PgStat_MsgTabstat *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Update database-wide stats.
- */
- dbentry->n_xact_commit += (PgStat_Counter) (msg->m_xact_commit);
- dbentry->n_xact_rollback += (PgStat_Counter) (msg->m_xact_rollback);
- dbentry->n_block_read_time += msg->m_block_read_time;
- dbentry->n_block_write_time += msg->m_block_write_time;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- PgStat_TableEntry *tabmsg = &(msg->m_entry[i]);
-
- tabentry = (PgStat_StatTabEntry *) hash_search(dbentry->tables,
- (void *) &(tabmsg->t_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new table entry, initialize counters to the values we
- * just got.
- */
- tabentry->numscans = tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned = tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched = tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted = tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated = tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted = tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated = tabmsg->t_counts.t_tuples_hot_updated;
- tabentry->n_live_tuples = tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples = tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze = tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum = tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched = tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit = tabmsg->t_counts.t_blocks_hit;
-
- tabentry->vacuum_timestamp = 0;
- tabentry->vacuum_count = 0;
- tabentry->autovac_vacuum_timestamp = 0;
- tabentry->autovac_vacuum_count = 0;
- tabentry->analyze_timestamp = 0;
- tabentry->analyze_count = 0;
- tabentry->autovac_analyze_timestamp = 0;
- tabentry->autovac_analyze_count = 0;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- tabentry->numscans += tabmsg->t_counts.t_numscans;
- tabentry->tuples_returned += tabmsg->t_counts.t_tuples_returned;
- tabentry->tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- tabentry->tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- tabentry->tuples_updated += tabmsg->t_counts.t_tuples_updated;
- tabentry->tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- tabentry->tuples_hot_updated += tabmsg->t_counts.t_tuples_hot_updated;
- /* If table was truncated, first reset the live/dead counters */
- if (tabmsg->t_counts.t_truncated)
- {
- tabentry->n_live_tuples = 0;
- tabentry->n_dead_tuples = 0;
- tabentry->inserts_since_vacuum = 0;
- }
- tabentry->n_live_tuples += tabmsg->t_counts.t_delta_live_tuples;
- tabentry->n_dead_tuples += tabmsg->t_counts.t_delta_dead_tuples;
- tabentry->changes_since_analyze += tabmsg->t_counts.t_changed_tuples;
- tabentry->inserts_since_vacuum += tabmsg->t_counts.t_tuples_inserted;
- tabentry->blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- tabentry->blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-
- /* Clamp n_live_tuples in case of negative delta_live_tuples */
- tabentry->n_live_tuples = Max(tabentry->n_live_tuples, 0);
- /* Likewise for n_dead_tuples */
- tabentry->n_dead_tuples = Max(tabentry->n_dead_tuples, 0);
-
- /*
- * Add per-table stats to the per-database entry, too.
- */
- dbentry->n_tuples_returned += tabmsg->t_counts.t_tuples_returned;
- dbentry->n_tuples_fetched += tabmsg->t_counts.t_tuples_fetched;
- dbentry->n_tuples_inserted += tabmsg->t_counts.t_tuples_inserted;
- dbentry->n_tuples_updated += tabmsg->t_counts.t_tuples_updated;
- dbentry->n_tuples_deleted += tabmsg->t_counts.t_tuples_deleted;
- dbentry->n_blocks_fetched += tabmsg->t_counts.t_blocks_fetched;
- dbentry->n_blocks_hit += tabmsg->t_counts.t_blocks_hit;
- }
-}
-
-
-/* ----------
- * pgstat_recv_tabpurge() -
- *
- * Arrange for dead table removal.
- * ----------
- */
-static void
-pgstat_recv_tabpurge(PgStat_MsgTabpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->tables)
- return;
-
- /*
- * Process all table entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->tables,
- (void *) &(msg->m_tableid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-
-/* ----------
- * pgstat_recv_dropdb() -
- *
- * Arrange for dead database removal
- * ----------
- */
-static void
-pgstat_recv_dropdb(PgStat_MsgDropdb *msg, int len)
-{
- Oid dbid = msg->m_databaseid;
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable.
- */
- dbentry = pgstat_get_db_entry(dbid, false);
-
- /*
- * If found, remove it (along with the db statfile).
- */
- if (dbentry)
- {
- char statfile[MAXPGPATH];
-
- get_dbstat_filename(false, false, dbid, statfile, MAXPGPATH);
-
- elog(DEBUG2, "removing stats file \"%s\"", statfile);
- unlink(statfile);
-
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- if (hash_search(pgStatDBHash,
- (void *) &dbid,
- HASH_REMOVE, NULL) == NULL)
- ereport(ERROR,
- (errmsg("database hash table corrupted during cleanup --- abort")));
- }
-}
-
-
-/* ----------
- * pgstat_recv_resetcounter() -
- *
- * Reset the statistics for the specified database.
- * ----------
- */
-static void
-pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Lookup the database in the hashtable. Nothing to do if not there.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /*
- * We simply throw away all the database's table entries by recreating a
- * new hash table for them.
- */
- if (dbentry->tables != NULL)
- hash_destroy(dbentry->tables);
- if (dbentry->functions != NULL)
- hash_destroy(dbentry->functions);
-
- dbentry->tables = NULL;
- dbentry->functions = NULL;
-
- /*
- * Reset database-level stats, too. This creates empty hash tables for
- * tables and functions.
- */
- reset_dbentry_counters(dbentry);
-}
-
-/* ----------
- * pgstat_recv_resetsharedcounter() -
- *
- * Reset some shared statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
-{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global background writer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
- {
- /* Reset the archiver statistics for the cluster. */
- memset(&archiverStats, 0, sizeof(archiverStats));
- archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_WAL)
- {
- /* Reset the WAL statistics for the cluster. */
- memset(&walStats, 0, sizeof(walStats));
- walStats.stat_reset_timestamp = GetCurrentTimestamp();
- }
-
- /*
- * Presumably the sender of this message validated the target, don't
- * complain here if it's not valid
- */
-}
-
-/* ----------
- * pgstat_recv_resetsinglecounter() -
- *
- * Reset a statistics for a single object
- * ----------
- */
-static void
-pgstat_recv_resetsinglecounter(PgStat_MsgResetsinglecounter *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- if (!dbentry)
- return;
-
- /* Set the reset timestamp for the whole database */
- dbentry->stat_reset_timestamp = GetCurrentTimestamp();
-
- /* Remove object if it exists, ignore it if not */
- if (msg->m_resettype == RESET_TABLE)
- (void) hash_search(dbentry->tables, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
- else if (msg->m_resettype == RESET_FUNCTION)
- (void) hash_search(dbentry->functions, (void *) &(msg->m_objectid),
- HASH_REMOVE, NULL);
-}
-
-/* ----------
- * pgstat_recv_resetslrucounter() -
- *
- * Reset some SLRU statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetslrucounter(PgStat_MsgResetslrucounter *msg, int len)
-{
- int i;
- TimestampTz ts = GetCurrentTimestamp();
-
- for (i = 0; i < SLRU_NUM_ELEMENTS; i++)
- {
- /* reset entry with the given index, or all entries (index is -1) */
- if ((msg->m_index == -1) || (msg->m_index == i))
- {
- memset(&slruStats[i], 0, sizeof(slruStats[i]));
- slruStats[i].stat_reset_timestamp = ts;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_resetreplslotcounter() -
- *
- * Reset some replication slot statistics of the cluster.
- * ----------
- */
-static void
-pgstat_recv_resetreplslotcounter(PgStat_MsgResetreplslotcounter *msg,
- int len)
-{
- int i;
- int idx = -1;
- TimestampTz ts;
-
- ts = GetCurrentTimestamp();
- if (msg->clearall)
- {
- for (i = 0; i < nReplSlotStats; i++)
- pgstat_reset_replslot(i, ts);
- }
- else
- {
- /* Get the index of replication slot statistics to reset */
- idx = pgstat_replslot_index(msg->m_slotname, false);
-
- /*
- * Nothing to do if the given slot entry is not found. This could
- * happen when the slot with the given name is removed and the
- * corresponding statistics entry is also removed before receiving the
- * reset message.
- */
- if (idx < 0)
- return;
-
- /* Reset the stats for the requested replication slot */
- pgstat_reset_replslot(idx, ts);
- }
-}
-
-
-/* ----------
- * pgstat_recv_autovac() -
- *
- * Process an autovacuum signaling message.
- * ----------
- */
-static void
-pgstat_recv_autovac(PgStat_MsgAutovacStart *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- /*
- * Store the last autovacuum time in the database's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->last_autovac_time = msg->m_start_time;
-}
-
-/* ----------
- * pgstat_recv_vacuum() -
- *
- * Process a VACUUM message.
- * ----------
- */
-static void
-pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * It is quite possible that a non-aggressive VACUUM ended up skipping
- * various pages, however, we'll zero the insert counter here regardless.
- * It's currently used only to track when we need to perform an "insert"
- * autovacuum, which are mainly intended to freeze newly inserted tuples.
- * Zeroing this may just mean we'll not try to vacuum the table again
- * until enough tuples have been inserted to trigger another insert
- * autovacuum. An anti-wraparound autovacuum will catch any persistent
- * stragglers.
- */
- tabentry->inserts_since_vacuum = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
- tabentry->autovac_vacuum_count++;
- }
- else
- {
- tabentry->vacuum_timestamp = msg->m_vacuumtime;
- tabentry->vacuum_count++;
- }
-}
-
-/* ----------
- * pgstat_recv_analyze() -
- *
- * Process an ANALYZE message.
- * ----------
- */
-static void
-pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- PgStat_StatTabEntry *tabentry;
-
- /*
- * Store the data in the table's hashtable entry.
- */
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- tabentry = pgstat_get_tab_entry(dbentry, msg->m_tableoid, true);
-
- tabentry->n_live_tuples = msg->m_live_tuples;
- tabentry->n_dead_tuples = msg->m_dead_tuples;
-
- /*
- * If commanded, reset changes_since_analyze to zero. This forgets any
- * changes that were committed while the ANALYZE was in progress, but we
- * have no good way to estimate how many of those there were.
- */
- if (msg->m_resetcounter)
- tabentry->changes_since_analyze = 0;
-
- if (msg->m_autovacuum)
- {
- tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
- tabentry->autovac_analyze_count++;
- }
- else
- {
- tabentry->analyze_timestamp = msg->m_analyzetime;
- tabentry->analyze_count++;
- }
-}
-
-
-/* ----------
- * pgstat_recv_archiver() -
- *
- * Process a ARCHIVER message.
- * ----------
- */
-static void
-pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
-{
- if (msg->m_failed)
- {
- /* Failed archival attempt */
- ++archiverStats.failed_count;
- memcpy(archiverStats.last_failed_wal, msg->m_xlog,
- sizeof(archiverStats.last_failed_wal));
- archiverStats.last_failed_timestamp = msg->m_timestamp;
- }
- else
- {
- /* Successful archival operation */
- ++archiverStats.archived_count;
- memcpy(archiverStats.last_archived_wal, msg->m_xlog,
- sizeof(archiverStats.last_archived_wal));
- archiverStats.last_archived_timestamp = msg->m_timestamp;
- }
-}
-
-/* ----------
- * pgstat_recv_bgwriter() -
- *
- * Process a BGWRITER message.
- * ----------
- */
-static void
-pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
-{
- globalStats.timed_checkpoints += msg->m_timed_checkpoints;
- globalStats.requested_checkpoints += msg->m_requested_checkpoints;
- globalStats.checkpoint_write_time += msg->m_checkpoint_write_time;
- globalStats.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.buf_written_clean += msg->m_buf_written_clean;
- globalStats.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.buf_written_backend += msg->m_buf_written_backend;
- globalStats.buf_fsync_backend += msg->m_buf_fsync_backend;
- globalStats.buf_alloc += msg->m_buf_alloc;
-}
-
-/* ----------
- * pgstat_recv_wal() -
- *
- * Process a WAL message.
- * ----------
- */
-static void
-pgstat_recv_wal(PgStat_MsgWal *msg, int len)
-{
- walStats.wal_records += msg->m_wal_records;
- walStats.wal_fpi += msg->m_wal_fpi;
- walStats.wal_bytes += msg->m_wal_bytes;
- walStats.wal_buffers_full += msg->m_wal_buffers_full;
-}
-
-/* ----------
- * pgstat_recv_slru() -
- *
- * Process a SLRU message.
- * ----------
- */
-static void
-pgstat_recv_slru(PgStat_MsgSLRU *msg, int len)
-{
- slruStats[msg->m_index].blocks_zeroed += msg->m_blocks_zeroed;
- slruStats[msg->m_index].blocks_hit += msg->m_blocks_hit;
- slruStats[msg->m_index].blocks_read += msg->m_blocks_read;
- slruStats[msg->m_index].blocks_written += msg->m_blocks_written;
- slruStats[msg->m_index].blocks_exists += msg->m_blocks_exists;
- slruStats[msg->m_index].flush += msg->m_flush;
- slruStats[msg->m_index].truncate += msg->m_truncate;
-}
-
-/* ----------
- * pgstat_recv_recoveryconflict() -
- *
- * Process a RECOVERYCONFLICT message.
- * ----------
- */
-static void
-pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- switch (msg->m_reason)
- {
- case PROCSIG_RECOVERY_CONFLICT_DATABASE:
-
- /*
- * Since we drop the information about the database as soon as it
- * replicates, there is no point in counting these conflicts.
- */
- break;
- case PROCSIG_RECOVERY_CONFLICT_TABLESPACE:
- dbentry->n_conflict_tablespace++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_LOCK:
- dbentry->n_conflict_lock++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT:
- dbentry->n_conflict_snapshot++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN:
- dbentry->n_conflict_bufferpin++;
- break;
- case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK:
- dbentry->n_conflict_startup_deadlock++;
- break;
- }
-}
-
-/* ----------
- * pgstat_recv_deadlock() -
- *
- * Process a DEADLOCK message.
- * ----------
- */
-static void
-pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_deadlocks++;
-}
-
-/* ----------
- * pgstat_recv_checksum_failure() -
- *
- * Process a CHECKSUMFAILURE message.
- * ----------
- */
-static void
-pgstat_recv_checksum_failure(PgStat_MsgChecksumFailure *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_checksum_failures += msg->m_failurecount;
- dbentry->last_checksum_failure = msg->m_failure_time;
-}
-
-/* ----------
- * pgstat_recv_replslot() -
- *
- * Process a REPLSLOT message.
- * ----------
- */
-static void
-pgstat_recv_replslot(PgStat_MsgReplSlot *msg, int len)
-{
- int idx;
-
- /*
- * Get the index of replication slot statistics. On dropping, we don't
- * create the new statistics.
- */
- idx = pgstat_replslot_index(msg->m_slotname, !msg->m_drop);
-
- /*
- * The slot entry is not found or there is no space to accommodate the new
- * entry. This could happen when the message for the creation of a slot
- * reached before the drop message even though the actual operations
- * happen in reverse order. In such a case, the next update of the
- * statistics for the same slot will create the required entry.
- */
- if (idx < 0)
- return;
-
- /* it must be a valid replication slot index */
- Assert(idx < nReplSlotStats);
-
- if (msg->m_drop)
- {
- /* Remove the replication slot statistics with the given name */
- if (idx < nReplSlotStats - 1)
- memcpy(&replSlotStats[idx], &replSlotStats[nReplSlotStats - 1],
- sizeof(PgStat_ReplSlotStats));
- nReplSlotStats--;
- }
- else
- {
- /* Update the replication slot statistics */
- replSlotStats[idx].spill_txns += msg->m_spill_txns;
- replSlotStats[idx].spill_count += msg->m_spill_count;
- replSlotStats[idx].spill_bytes += msg->m_spill_bytes;
- replSlotStats[idx].stream_txns += msg->m_stream_txns;
- replSlotStats[idx].stream_count += msg->m_stream_count;
- replSlotStats[idx].stream_bytes += msg->m_stream_bytes;
- }
-}
-
-/* ----------
- * pgstat_recv_tempfile() -
- *
- * Process a TEMPFILE message.
- * ----------
- */
-static void
-pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- dbentry->n_temp_bytes += msg->m_filesize;
- dbentry->n_temp_files += 1;
-}
-
-/* ----------
- * pgstat_recv_funcstat() -
- *
- * Count what the backend has done.
- * ----------
- */
-static void
-pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len)
-{
- PgStat_FunctionEntry *funcmsg = &(msg->m_entry[0]);
- PgStat_StatDBEntry *dbentry;
- PgStat_StatFuncEntry *funcentry;
- int i;
- bool found;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, true);
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++, funcmsg++)
- {
- funcentry = (PgStat_StatFuncEntry *) hash_search(dbentry->functions,
- (void *) &(funcmsg->f_id),
- HASH_ENTER, &found);
-
- if (!found)
- {
- /*
- * If it's a new function entry, initialize counters to the values
- * we just got.
- */
- funcentry->f_numcalls = funcmsg->f_numcalls;
- funcentry->f_total_time = funcmsg->f_total_time;
- funcentry->f_self_time = funcmsg->f_self_time;
- }
- else
- {
- /*
- * Otherwise add the values to the existing entry.
- */
- funcentry->f_numcalls += funcmsg->f_numcalls;
- funcentry->f_total_time += funcmsg->f_total_time;
- funcentry->f_self_time += funcmsg->f_self_time;
- }
- }
-}
-
-/* ----------
- * pgstat_recv_funcpurge() -
- *
- * Arrange for dead function removal.
- * ----------
- */
-static void
-pgstat_recv_funcpurge(PgStat_MsgFuncpurge *msg, int len)
-{
- PgStat_StatDBEntry *dbentry;
- int i;
-
- dbentry = pgstat_get_db_entry(msg->m_databaseid, false);
-
- /*
- * No need to purge if we don't even know the database.
- */
- if (!dbentry || !dbentry->functions)
- return;
-
- /*
- * Process all function entries in the message.
- */
- for (i = 0; i < msg->m_nentries; i++)
- {
- /* Remove from hashtable if present; we don't care if it's not. */
- (void) hash_search(dbentry->functions,
- (void *) &(msg->m_functionid[i]),
- HASH_REMOVE, NULL);
- }
-}
-
-/* ----------
- * pgstat_write_statsfile_needed() -
- *
- * Do we need to write out any stats files?
- * ----------
- */
-static bool
-pgstat_write_statsfile_needed(void)
-{
- if (pending_write_requests != NIL)
- return true;
-
- /* Everything was written recently */
- return false;
-}
-
-/* ----------
- * pgstat_db_requested() -
- *
- * Checks whether stats for a particular DB need to be written to a file.
- * ----------
- */
-static bool
-pgstat_db_requested(Oid databaseid)
-{
- /*
- * If any requests are outstanding at all, we should write the stats for
- * shared catalogs (the "database" with OID 0). This ensures that
- * backends will see up-to-date stats for shared catalogs, even though
- * they send inquiry messages mentioning only their own DB.
- */
- if (databaseid == InvalidOid && pending_write_requests != NIL)
- return true;
-
- /* Search to see if there's an open request to write this database. */
- if (list_member_oid(pending_write_requests, databaseid))
- return true;
-
- return false;
+ /* Reset variables */
+ pgStatLocalContext = NULL;
+ localBackendStatusTable = NULL;
+ localNumBackends = 0;
+ }
+
+ /* Invalidate the simple cache keys */
+ cached_dbent_key = stathashkey_zero;
+ cached_tabent_key = stathashkey_zero;
+ cached_funcent_key = stathashkey_zero;
+ cached_archiverstats_is_valid = false;
+ cached_bgwriterstats_is_valid = false;
+ cached_checkpointerstats_is_valid = false;
+ cached_walstats_is_valid = false;
+ cached_slrustats_is_valid = false;
+ n_cached_replslotstats = -1;
}
/*
@@ -7126,60 +5942,6 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
-/* ----------
- * pgstat_replslot_index
- *
- * Return the index of entry of a replication slot with the given name, or
- * -1 if the slot is not found.
- *
- * create_it tells whether to create the new slot entry if it is not found.
- * ----------
- */
-static int
-pgstat_replslot_index(const char *name, bool create_it)
-{
- int i;
-
- Assert(nReplSlotStats <= max_replication_slots);
- for (i = 0; i < nReplSlotStats; i++)
- {
- if (strcmp(replSlotStats[i].slotname, name) == 0)
- return i; /* found */
- }
-
- /*
- * The slot is not found. We don't want to register the new statistics if
- * the list is already full or the caller didn't request.
- */
- if (i == max_replication_slots || !create_it)
- return -1;
-
- /* Register new slot */
- memset(&replSlotStats[nReplSlotStats], 0, sizeof(PgStat_ReplSlotStats));
- strlcpy(replSlotStats[nReplSlotStats].slotname, name, NAMEDATALEN);
-
- return nReplSlotStats++;
-}
-
-/* ----------
- * pgstat_reset_replslot
- *
- * Reset the replication slot stats at index 'i'.
- * ----------
- */
-static void
-pgstat_reset_replslot(int i, TimestampTz ts)
-{
- /* reset only counters. Don't clear slot name */
- replSlotStats[i].spill_txns = 0;
- replSlotStats[i].spill_count = 0;
- replSlotStats[i].spill_bytes = 0;
- replSlotStats[i].stream_txns = 0;
- replSlotStats[i].stream_count = 0;
- replSlotStats[i].stream_bytes = 0;
- replSlotStats[i].stat_reset_timestamp = ts;
-}
-
/*
* pgstat_slru_index
*
@@ -7224,7 +5986,7 @@ pgstat_slru_name(int slru_idx)
* Returns pointer to entry with counters for given SLRU (based on the name
* stored in SlruCtl as lwlock tranche name).
*/
-static inline PgStat_MsgSLRU *
+static PgStat_SLRUStats *
slru_entry(int slru_idx)
{
/*
@@ -7235,7 +5997,7 @@ slru_entry(int slru_idx)
Assert((slru_idx >= 0) && (slru_idx < SLRU_NUM_ELEMENTS));
- return &SLRUStats[slru_idx];
+ return &local_SLRUStats[slru_idx];
}
/*
@@ -7245,41 +6007,48 @@ slru_entry(int slru_idx)
void
pgstat_count_slru_page_zeroed(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_zeroed += 1;
+ slru_entry(slru_idx)->blocks_zeroed += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_hit(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_hit += 1;
+ slru_entry(slru_idx)->blocks_hit += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_exists(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_exists += 1;
+ slru_entry(slru_idx)->blocks_exists += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_read(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_read += 1;
+ slru_entry(slru_idx)->blocks_read += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_page_written(int slru_idx)
{
- slru_entry(slru_idx)->m_blocks_written += 1;
+ slru_entry(slru_idx)->blocks_written += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_flush(int slru_idx)
{
- slru_entry(slru_idx)->m_flush += 1;
+ slru_entry(slru_idx)->flush += 1;
+ have_slrustats = true;
}
void
pgstat_count_slru_truncate(int slru_idx)
{
- slru_entry(slru_idx)->m_truncate += 1;
+ slru_entry(slru_idx)->truncate += 1;
+ have_slrustats = true;
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index af91c313e2..9b9c9b1c11 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -251,7 +251,6 @@ static pid_t StartupPID = 0,
WalReceiverPID = 0,
AutoVacPID = 0,
PgArchPID = 0,
- PgStatPID = 0,
SysLoggerPID = 0;
/* Startup process's status */
@@ -512,7 +511,6 @@ typedef struct
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
PMSignalData *PMSignalState;
- InheritableSocket pgStatSock;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -1327,12 +1325,6 @@ PostmasterMain(int argc, char *argv[])
*/
RemovePgTempFiles();
- /*
- * Initialize stats collection subsystem (this does NOT start the
- * collector process!)
- */
- pgstat_init();
-
/*
* Initialize the autovacuum subsystem (again, no process start yet)
*/
@@ -1782,11 +1774,6 @@ ServerLoop(void)
start_autovac_launcher = false; /* signal processed */
}
- /* If we have lost the stats collector, try to start a new one */
- if (PgStatPID == 0 &&
- (pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
-
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
PgArchPID = StartArchiver();
@@ -2699,8 +2686,6 @@ SIGHUP_handler(SIGNAL_ARGS)
signal_child(PgArchPID, SIGHUP);
if (SysLoggerPID != 0)
signal_child(SysLoggerPID, SIGHUP);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGHUP);
/* Reload authentication config files too */
if (!load_hba())
@@ -3029,8 +3014,6 @@ reaper(SIGNAL_ARGS)
AutoVacPID = StartAutoVacLauncher();
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartArchiver();
- if (PgStatPID == 0)
- PgStatPID = pgstat_start();
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3097,13 +3080,6 @@ reaper(SIGNAL_ARGS)
SignalChildren(SIGUSR2);
pmState = PM_SHUTDOWN_2;
-
- /*
- * We can also shut down the stats collector now; there's
- * nothing left for it to do.
- */
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
else
{
@@ -3176,22 +3152,6 @@ reaper(SIGNAL_ARGS)
continue;
}
- /*
- * Was it the statistics collector? If so, just try to start a new
- * one; no need to force reset of the rest of the system. (If fail,
- * we'll try again in future cycles of the main loop.)
- */
- if (pid == PgStatPID)
- {
- PgStatPID = 0;
- if (!EXIT_STATUS_0(exitstatus))
- LogChildExit(LOG, _("statistics collector process"),
- pid, exitstatus);
- if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
- continue;
- }
-
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
{
@@ -3653,22 +3613,6 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
signal_child(PgArchPID, SIGQUIT);
}
- /*
- * Force a power-cycle of the pgstat process too. (This isn't absolutely
- * necessary, but it seems like a good idea for robustness, and it
- * simplifies the state-machine logic in the case where a shutdown request
- * arrives during crash processing.)
- */
- if (PgStatPID != 0 && take_action)
- {
- ereport(DEBUG2,
- (errmsg_internal("sending %s to process %d",
- "SIGQUIT",
- (int) PgStatPID)));
- signal_child(PgStatPID, SIGQUIT);
- allow_immediate_pgstat_restart();
- }
-
/* We do NOT restart the syslogger */
if (Shutdown != ImmediateShutdown)
@@ -3895,8 +3839,6 @@ PostmasterStateMachine(void)
SignalChildren(SIGQUIT);
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
- if (PgStatPID != 0)
- signal_child(PgStatPID, SIGQUIT);
}
}
}
@@ -3920,8 +3862,7 @@ PostmasterStateMachine(void)
{
/*
* PM_WAIT_DEAD_END state ends when the BackendList is entirely empty
- * (ie, no dead_end children remain), and the archiver and stats
- * collector are gone too.
+ * (ie, no dead_end children remain), and the archiveris gone too.
*
* The reason we wait for those two is to protect them against a new
* postmaster starting conflicting subprocesses; this isn't an
@@ -3931,8 +3872,7 @@ PostmasterStateMachine(void)
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
*/
- if (dlist_is_empty(&BackendList) &&
- PgArchPID == 0 && PgStatPID == 0)
+ if (dlist_is_empty(&BackendList) && PgArchPID == 0)
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
@@ -4133,8 +4073,6 @@ TerminateChildren(int signal)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
- if (PgStatPID != 0)
- signal_child(PgStatPID, signal);
}
/*
@@ -5067,18 +5005,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5191,12 +5117,6 @@ sigusr1_handler(SIGNAL_ARGS)
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
- /*
- * Likewise, start other special children as needed.
- */
- Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
-
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -6113,7 +6033,6 @@ extern slock_t *ShmemLock;
extern slock_t *ProcStructLock;
extern PGPROC *AuxiliaryProcs;
extern PMSignalData *PMSignalState;
-extern pgsocket pgStatSock;
extern pg_time_t first_syslogger_file_time;
#ifndef WIN32
@@ -6169,8 +6088,6 @@ save_backend_variables(BackendParameters *param, Port *port,
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- if (!write_inheritable_socket(¶m->pgStatSock, pgStatSock, childPid))
- return false;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -6403,7 +6320,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- read_inheritable_socket(&pgStatSock, ¶m->pgStatSock);
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 0f54635550..d21801cf90 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1568,8 +1568,8 @@ is_checksummed_file(const char *fullpath, const char *filename)
*
* If 'missing_ok' is true, will not throw an error if the file is not found.
*
- * If dboid is anything other than InvalidOid then any checksum failures detected
- * will get reported to the stats collector.
+ * If dboid is anything other than InvalidOid then any checksum failures
+ * detected will get reported to the activity stats facility.
*
* Returns true if the file was successfully sent, false if 'missing_ok',
* and the file did not exist.
diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c
index e00c7ffc01..608818beea 100644
--- a/src/backend/replication/slot.c
+++ b/src/backend/replication/slot.c
@@ -692,14 +692,10 @@ ReplicationSlotDropPtr(ReplicationSlot *slot)
(errmsg("could not remove directory \"%s\"", tmppath)));
/*
- * Send a message to drop the replication slot to the stats collector.
- * Since there is no guarantee of the order of message transfer on a UDP
- * connection, it's possible that a message for creating a new slot
- * reaches before a message for removing the old slot. We send the drop
- * and create messages while holding ReplicationSlotAllocationLock to
- * reduce that possibility. If the messages reached in reverse, we would
- * lose one statistics update message. But the next update message will
- * create the statistics for the replication slot.
+ * Drop the statistics entry for the replication slot. Do this while
+ * holding ReplicationSlotAllocationLock so that we don't drop a statistics
+ * entry for another slot with the same name just created on another
+ * session.
*/
if (SlotIsLogical(slot))
pgstat_report_replslot_drop(NameStr(slot->data.name));
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 71b5852224..160977d50d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2047,7 +2047,7 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- BgWriterStats.m_buf_written_checkpoints++;
+ CheckPointerStats.buf_written_checkpoints++;
num_written++;
}
}
@@ -2157,7 +2157,7 @@ BgBufferSync(WritebackContext *wb_context)
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
/* Report buffer alloc counts to pgstat */
- BgWriterStats.m_buf_alloc += recent_alloc;
+ BgWriterStats.buf_alloc += recent_alloc;
/*
* If we're not running the LRU scan, just stop after doing the stats
@@ -2347,7 +2347,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
if (++num_written >= bgwriter_lru_maxpages)
{
- BgWriterStats.m_maxwritten_clean++;
+ BgWriterStats.maxwritten_clean++;
break;
}
}
@@ -2355,7 +2355,7 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- BgWriterStats.m_buf_written_clean += num_written;
+ BgWriterStats.buf_written_clean += num_written;
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index f9bbe97b50..78cfe91eab 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -149,6 +149,7 @@ CreateSharedMemoryAndSemaphores(void)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, StatsShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -267,6 +268,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ StatsShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index db7e59f8b7..e6e4c0fb04 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -177,7 +177,9 @@ static const char *const BuiltinTrancheNames[] = {
/* LWTRANCHE_PARALLEL_APPEND: */
"ParallelAppend",
/* LWTRANCHE_PER_XACT_PREDICATE_LIST: */
- "PerXactPredicateList"
+ "PerXactPredicateList",
+ /* LWTRANCHE_STATS: */
+ "ActivityStatistics"
};
StaticAssertDecl(lengthof(BuiltinTrancheNames) ==
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index 774292fd94..91bf8d5b5d 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -53,3 +53,4 @@ XactTruncationLock 44
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+StatsLock 48
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 0f31ff3822..fc96f21519 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -414,8 +414,8 @@ smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo)
DropRelFileNodesAllBuffers(rnodes, nrels);
/*
- * It'd be nice to tell the stats collector to forget them immediately,
- * too. But we can't because we don't know the OIDs.
+ * It'd be nice to tell the activity stats facility to forget them
+ * immediately, too. But we can't because we don't know the OIDs.
*/
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 28055680aa..8b51852505 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3271,6 +3271,12 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (IdleStatsUpdateTimeoutPending)
+ {
+ IdleStatsUpdateTimeoutPending = false;
+ pgstat_report_stat(true);
+ }
}
@@ -3842,6 +3848,7 @@ PostgresMain(int argc, char *argv[],
volatile bool send_ready_for_query = true;
bool idle_in_transaction_timeout_enabled = false;
bool idle_session_timeout_enabled = false;
+ bool idle_stats_update_timeout_enabled = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4238,11 +4245,12 @@ PostgresMain(int argc, char *argv[],
* Note: this includes fflush()'ing the last of the prior output.
*
* This is also a good time to send collected statistics to the
- * collector, and to update the PS stats display. We avoid doing
- * those every time through the message loop because it'd slow down
- * processing of batched messages, and because we don't want to report
- * uncommitted updates (that confuses autovacuum). The notification
- * processor wants a call too, if we are not in a transaction block.
+ * activity statistics, and to update the PS stats display. We avoid
+ * doing those every time through the message loop because it'd slow
+ * down processing of batched messages, and because we don't want to
+ * report uncommitted updates (that confuses autovacuum). The
+ * notification processor wants a call too, if we are not in a
+ * transaction block.
*
* Also, if an idle timeout is enabled, start the timer for that.
*/
@@ -4276,6 +4284,8 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long stats_timeout;
+
/* Send out notify signals and transmit self-notifies */
ProcessCompletedNotifies();
@@ -4288,8 +4298,14 @@ PostgresMain(int argc, char *argv[],
if (notifyInterruptPending)
ProcessNotifyInterrupt();
- pgstat_report_stat(false);
-
+ /* Start the idle-stats-update timer */
+ stats_timeout = pgstat_report_stat(false);
+ if (stats_timeout > 0)
+ {
+ idle_stats_update_timeout_enabled = true;
+ enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT,
+ stats_timeout);
+ }
set_ps_display("idle");
pgstat_report_activity(STATE_IDLE, NULL);
@@ -4323,9 +4339,9 @@ PostgresMain(int argc, char *argv[],
firstchar = ReadCommand(&input_message);
/*
- * (4) turn off the idle-in-transaction and idle-session timeouts, if
- * active. We do this before step (5) so that any last-moment timeout
- * is certain to be detected in step (5).
+ * (4) turn off the idle-in-transaction, idle-session and
+ * idle-state-update timeouts if active. We do this before step (5) so
+ * that any last-moment timeout is certain to be detected in step (5).
*
* At most one of these timeouts will be active, so there's no need to
* worry about combining the timeout.c calls into one.
@@ -4340,6 +4356,11 @@ PostgresMain(int argc, char *argv[],
disable_timeout(IDLE_SESSION_TIMEOUT, false);
idle_session_timeout_enabled = false;
}
+ if (idle_stats_update_timeout_enabled)
+ {
+ disable_timeout(IDLE_STATS_UPDATE_TIMEOUT, false);
+ idle_stats_update_timeout_enabled = false;
+ }
/*
* (5) disable async signal conditions again.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5c12a165a1..fd6465dfb9 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1,7 +1,7 @@
/*-------------------------------------------------------------------------
*
* pgstatfuncs.c
- * Functions for accessing the statistics collector data
+ * Functions for accessing the activity statistics data
*
* Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -35,9 +35,6 @@
#define HAS_PGSTAT_PERMISSIONS(role) (is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS) || has_privs_of_role(GetUserId(), role))
-/* Global bgwriter statistics, from bgwriter.c */
-extern PgStat_MsgBgWriter bgwriterStats;
-
Datum
pg_stat_get_numscans(PG_FUNCTION_ARGS)
{
@@ -1269,7 +1266,7 @@ pg_stat_get_db_xact_commit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_commit);
+ result = (int64) (dbentry->counts.n_xact_commit);
PG_RETURN_INT64(result);
}
@@ -1285,7 +1282,7 @@ pg_stat_get_db_xact_rollback(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_xact_rollback);
+ result = (int64) (dbentry->counts.n_xact_rollback);
PG_RETURN_INT64(result);
}
@@ -1301,7 +1298,7 @@ pg_stat_get_db_blocks_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_fetched);
+ result = (int64) (dbentry->counts.n_blocks_fetched);
PG_RETURN_INT64(result);
}
@@ -1317,7 +1314,7 @@ pg_stat_get_db_blocks_hit(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_blocks_hit);
+ result = (int64) (dbentry->counts.n_blocks_hit);
PG_RETURN_INT64(result);
}
@@ -1333,7 +1330,7 @@ pg_stat_get_db_tuples_returned(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_returned);
+ result = (int64) (dbentry->counts.n_tuples_returned);
PG_RETURN_INT64(result);
}
@@ -1349,7 +1346,7 @@ pg_stat_get_db_tuples_fetched(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_fetched);
+ result = (int64) (dbentry->counts.n_tuples_fetched);
PG_RETURN_INT64(result);
}
@@ -1365,7 +1362,7 @@ pg_stat_get_db_tuples_inserted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_inserted);
+ result = (int64) (dbentry->counts.n_tuples_inserted);
PG_RETURN_INT64(result);
}
@@ -1381,7 +1378,7 @@ pg_stat_get_db_tuples_updated(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_updated);
+ result = (int64) (dbentry->counts.n_tuples_updated);
PG_RETURN_INT64(result);
}
@@ -1397,7 +1394,7 @@ pg_stat_get_db_tuples_deleted(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_tuples_deleted);
+ result = (int64) (dbentry->counts.n_tuples_deleted);
PG_RETURN_INT64(result);
}
@@ -1430,7 +1427,7 @@ pg_stat_get_db_temp_files(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_files;
+ result = dbentry->counts.n_temp_files;
PG_RETURN_INT64(result);
}
@@ -1446,7 +1443,7 @@ pg_stat_get_db_temp_bytes(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = dbentry->n_temp_bytes;
+ result = dbentry->counts.n_temp_bytes;
PG_RETURN_INT64(result);
}
@@ -1461,7 +1458,7 @@ pg_stat_get_db_conflict_tablespace(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace);
+ result = (int64) (dbentry->counts.n_conflict_tablespace);
PG_RETURN_INT64(result);
}
@@ -1476,7 +1473,7 @@ pg_stat_get_db_conflict_lock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_lock);
+ result = (int64) (dbentry->counts.n_conflict_lock);
PG_RETURN_INT64(result);
}
@@ -1491,7 +1488,7 @@ pg_stat_get_db_conflict_snapshot(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_snapshot);
+ result = (int64) (dbentry->counts.n_conflict_snapshot);
PG_RETURN_INT64(result);
}
@@ -1506,7 +1503,7 @@ pg_stat_get_db_conflict_bufferpin(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_bufferpin);
+ result = (int64) (dbentry->counts.n_conflict_bufferpin);
PG_RETURN_INT64(result);
}
@@ -1521,7 +1518,7 @@ pg_stat_get_db_conflict_startup_deadlock(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1536,11 +1533,11 @@ pg_stat_get_db_conflict_all(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_conflict_tablespace +
- dbentry->n_conflict_lock +
- dbentry->n_conflict_snapshot +
- dbentry->n_conflict_bufferpin +
- dbentry->n_conflict_startup_deadlock);
+ result = (int64) (dbentry->counts.n_conflict_tablespace +
+ dbentry->counts.n_conflict_lock +
+ dbentry->counts.n_conflict_snapshot +
+ dbentry->counts.n_conflict_bufferpin +
+ dbentry->counts.n_conflict_startup_deadlock);
PG_RETURN_INT64(result);
}
@@ -1555,7 +1552,7 @@ pg_stat_get_db_deadlocks(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_deadlocks);
+ result = (int64) (dbentry->counts.n_deadlocks);
PG_RETURN_INT64(result);
}
@@ -1573,7 +1570,7 @@ pg_stat_get_db_checksum_failures(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = (int64) (dbentry->n_checksum_failures);
+ result = (int64) (dbentry->counts.n_checksum_failures);
PG_RETURN_INT64(result);
}
@@ -1610,7 +1607,7 @@ pg_stat_get_db_blk_read_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_read_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_read_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1626,7 +1623,7 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
if ((dbentry = pgstat_fetch_stat_dbentry(dbid)) == NULL)
result = 0;
else
- result = ((double) dbentry->n_block_write_time) / 1000.0;
+ result = ((double) dbentry->counts.n_block_write_time) / 1000.0;
PG_RETURN_FLOAT8(result);
}
@@ -1634,69 +1631,71 @@ pg_stat_get_db_blk_write_time(PG_FUNCTION_ARGS)
Datum
pg_stat_get_bgwriter_timed_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->timed_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->timed_checkpoints);
}
Datum
pg_stat_get_bgwriter_requested_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->requested_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->requested_checkpoints);
}
Datum
pg_stat_get_bgwriter_buf_written_checkpoints(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_checkpoints);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_checkpoints);
}
Datum
pg_stat_get_bgwriter_buf_written_clean(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_clean);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_written_clean);
}
Datum
pg_stat_get_bgwriter_maxwritten_clean(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->maxwritten_clean);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->maxwritten_clean);
}
Datum
pg_stat_get_checkpoint_write_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
- PG_RETURN_FLOAT8((double) pgstat_fetch_global()->checkpoint_write_time);
+ PG_RETURN_FLOAT8((double)
+ pgstat_fetch_stat_checkpointer()->checkpoint_write_time);
}
Datum
pg_stat_get_checkpoint_sync_time(PG_FUNCTION_ARGS)
{
/* time is already in msec, just convert to double for presentation */
- PG_RETURN_FLOAT8((double) pgstat_fetch_global()->checkpoint_sync_time);
+ PG_RETURN_FLOAT8((double)
+ pgstat_fetch_stat_checkpointer()->checkpoint_sync_time);
}
Datum
pg_stat_get_bgwriter_stat_reset_time(PG_FUNCTION_ARGS)
{
- PG_RETURN_TIMESTAMPTZ(pgstat_fetch_global()->stat_reset_timestamp);
+ PG_RETURN_TIMESTAMPTZ(pgstat_fetch_stat_bgwriter()->stat_reset_timestamp);
}
Datum
pg_stat_get_buf_written_backend(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_written_backend);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_backend);
}
Datum
pg_stat_get_buf_fsync_backend(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_fsync_backend);
+ PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_fsync_backend);
}
Datum
pg_stat_get_buf_alloc(PG_FUNCTION_ARGS)
{
- PG_RETURN_INT64(pgstat_fetch_global()->buf_alloc);
+ PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_alloc);
}
/*
@@ -1710,7 +1709,7 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_WAL_COLS];
bool nulls[PG_STAT_GET_WAL_COLS];
char buf[256];
- PgStat_WalStats *wal_stats;
+ PgStat_Wal *wal_stats;
/* Initialise values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
@@ -1735,11 +1734,11 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
wal_stats = pgstat_fetch_stat_wal();
/* Fill values and NULLs */
- values[0] = Int64GetDatum(wal_stats->wal_records);
- values[1] = Int64GetDatum(wal_stats->wal_fpi);
+ values[0] = Int64GetDatum(wal_stats->wal_usage.wal_records);
+ values[1] = Int64GetDatum(wal_stats->wal_usage.wal_fpi);
/* Convert to numeric. */
- snprintf(buf, sizeof buf, UINT64_FORMAT, wal_stats->wal_bytes);
+ snprintf(buf, sizeof buf, UINT64_FORMAT, wal_stats->wal_usage.wal_bytes);
values[2] = DirectFunctionCall3(numeric_in,
CStringGetDatum(buf),
ObjectIdGetDatum(0),
@@ -2020,7 +2019,7 @@ pg_stat_get_xact_function_self_time(PG_FUNCTION_ARGS)
Datum
pg_stat_get_snapshot_timestamp(PG_FUNCTION_ARGS)
{
- PG_RETURN_TIMESTAMPTZ(pgstat_fetch_global()->stats_timestamp);
+ PG_RETURN_TIMESTAMPTZ(pgstat_get_stat_timestamp());
}
/* Discard the active statistics snapshot */
@@ -2108,7 +2107,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDesc tupdesc;
Datum values[7];
bool nulls[7];
- PgStat_ArchiverStats *archiver_stats;
+ PgStat_Archiver *archiver_stats;
/* Initialise values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
@@ -2178,7 +2177,7 @@ pg_stat_get_replication_slots(PG_FUNCTION_ARGS)
Tuplestorestate *tupstore;
MemoryContext per_query_ctx;
MemoryContext oldcontext;
- PgStat_ReplSlotStats *slotstats;
+ PgStat_ReplSlot *slotstats;
int nstats;
int i;
@@ -2211,7 +2210,7 @@ pg_stat_get_replication_slots(PG_FUNCTION_ARGS)
{
Datum values[PG_STAT_GET_REPLICATION_SLOT_COLS];
bool nulls[PG_STAT_GET_REPLICATION_SLOT_COLS];
- PgStat_ReplSlotStats *s = &(slotstats[i]);
+ PgStat_ReplSlot *s = &(slotstats[i]);
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..0762c2034c 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -72,6 +72,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/optimizer.h"
+#include "pgstat.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rowsecurity.h"
#include "storage/lmgr.h"
@@ -2366,6 +2367,10 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
*/
RelationCloseSmgr(relation);
+ /* break mutual link with stats entry */
+ pgstat_delinkstats(relation);
+
+ if (relation->rd_rel)
/*
* Free all the subsidiary data structures of the relcache entry, then the
* entry itself.
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index ea28769d6a..997afcab6d 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -34,6 +34,7 @@ volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 0f67b99cc5..2567668b6c 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -269,9 +269,6 @@ GetBackendTypeDesc(BackendType backendType)
case B_ARCHIVER:
backendDesc = "archiver";
break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
case B_LOGGER:
backendDesc = "logger";
break;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e5965bc517..d4c17fd7ab 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
static void IdleSessionTimeoutHandler(void);
+static void IdleStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -621,6 +622,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
RegisterTimeout(IDLE_SESSION_TIMEOUT, IdleSessionTimeoutHandler);
+ RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT,
+ IdleStatsUpdateTimeoutHandler);
}
/*
@@ -1243,6 +1246,14 @@ IdleSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleStatsUpdateTimeoutHandler(void)
+{
+ IdleStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 17579eeaca..85299e2138 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -746,8 +746,8 @@ const char *const config_group_names[] =
gettext_noop("Statistics"),
/* STATS_MONITORING */
gettext_noop("Statistics / Monitoring"),
- /* STATS_COLLECTOR */
- gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* STATS_ACTIVITY */
+ gettext_noop("Statistics / Query and Index Activity Statistics"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
@@ -1457,7 +1457,7 @@ static struct config_bool ConfigureNamesBool[] =
#endif
{
- {"track_activities", PGC_SUSET, STATS_COLLECTOR,
+ {"track_activities", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects information about executing commands."),
gettext_noop("Enables the collection of information on the currently "
"executing command of each session, along with "
@@ -1468,7 +1468,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_counts", PGC_SUSET, STATS_COLLECTOR,
+ {"track_counts", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects statistics on database activity."),
NULL
},
@@ -1477,7 +1477,7 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
- {"track_io_timing", PGC_SUSET, STATS_COLLECTOR,
+ {"track_io_timing", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects timing statistics for database I/O activity."),
NULL
},
@@ -4356,7 +4356,7 @@ static struct config_string ConfigureNamesString[] =
},
{
- {"stats_temp_directory", PGC_SIGHUP, STATS_COLLECTOR,
+ {"stats_temp_directory", PGC_SIGHUP, STATS_ACTIVITY,
gettext_noop("Writes temporary statistics files to the specified directory."),
NULL,
GUC_SUPERUSER_ONLY
@@ -4692,7 +4692,7 @@ static struct config_enum ConfigureNamesEnum[] =
},
{
- {"track_functions", PGC_SUSET, STATS_COLLECTOR,
+ {"track_functions", PGC_SUSET, STATS_ACTIVITY,
gettext_noop("Collects function-level statistics on database activity."),
NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 8930a94fff..4f5b6bdb12 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -580,7 +580,7 @@
# STATISTICS
#------------------------------------------------------------------------------
-# - Query and Index Statistics Collector -
+# - Query and Index Activity Statistics -
#track_activities = on
#track_counts = on
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index f674a7c94e..c61b828bf3 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -6,7 +6,7 @@ use File::Basename qw(basename dirname);
use File::Path qw(rmtree);
use PostgresNode;
use TestLib;
-use Test::More tests => 109;
+use Test::More tests => 108;
program_help_ok('pg_basebackup');
program_version_ok('pg_basebackup');
@@ -124,7 +124,7 @@ is_deeply(
# Contents of these directories should not be copied.
foreach my $dirname (
- qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_stat_tmp pg_subtrans)
+ qw(pg_dynshmem pg_notify pg_replslot pg_serial pg_snapshots pg_subtrans)
)
{
is_deeply(
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index adb9f819bb..12708b9470 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -84,6 +84,8 @@ extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
@@ -322,7 +324,6 @@ typedef enum BackendType
B_WAL_SENDER,
B_WAL_WRITER,
B_ARCHIVER,
- B_STATS_COLLECTOR,
B_LOGGER,
} BackendType;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c38b689710..0472b728bf 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1,7 +1,7 @@
/* ----------
* pgstat.h
*
- * Definitions for the PostgreSQL statistics collector daemon.
+ * Definitions for the PostgreSQL activity statistics facility.
*
* Copyright (c) 2001-2021, PostgreSQL Global Development Group
*
@@ -12,12 +12,15 @@
#define PGSTAT_H
#include "datatype/timestamp.h"
+#include "executor/instrument.h"
#include "libpq/pqcomm.h"
#include "miscadmin.h"
#include "port/atomics.h"
+#include "lib/dshash.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
+#include "storage/lwlock.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -27,8 +30,8 @@
* ----------
*/
#define PGSTAT_STAT_PERMANENT_DIRECTORY "pg_stat"
-#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
-#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
+#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/saved_stats"
+#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/saved_stats.tmp"
/* Default directory to store temporary statistics data in */
#define PG_STAT_TMP_DIR "pg_stat_tmp"
@@ -41,38 +44,6 @@ typedef enum TrackFunctionsLevel
TRACK_FUNC_ALL
} TrackFunctionsLevel;
-/* ----------
- * The types of backend -> collector messages
- * ----------
- */
-typedef enum StatMsgType
-{
- PGSTAT_MTYPE_DUMMY,
- PGSTAT_MTYPE_INQUIRY,
- PGSTAT_MTYPE_TABSTAT,
- PGSTAT_MTYPE_TABPURGE,
- PGSTAT_MTYPE_DROPDB,
- PGSTAT_MTYPE_RESETCOUNTER,
- PGSTAT_MTYPE_RESETSHAREDCOUNTER,
- PGSTAT_MTYPE_RESETSINGLECOUNTER,
- PGSTAT_MTYPE_RESETSLRUCOUNTER,
- PGSTAT_MTYPE_RESETREPLSLOTCOUNTER,
- PGSTAT_MTYPE_AUTOVAC_START,
- PGSTAT_MTYPE_VACUUM,
- PGSTAT_MTYPE_ANALYZE,
- PGSTAT_MTYPE_ARCHIVER,
- PGSTAT_MTYPE_BGWRITER,
- PGSTAT_MTYPE_WAL,
- PGSTAT_MTYPE_SLRU,
- PGSTAT_MTYPE_FUNCSTAT,
- PGSTAT_MTYPE_FUNCPURGE,
- PGSTAT_MTYPE_RECOVERYCONFLICT,
- PGSTAT_MTYPE_TEMPFILE,
- PGSTAT_MTYPE_DEADLOCK,
- PGSTAT_MTYPE_CHECKSUMFAILURE,
- PGSTAT_MTYPE_REPLSLOT,
-} StatMsgType;
-
/* ----------
* The data type used for counters.
* ----------
@@ -83,9 +54,8 @@ typedef int64 PgStat_Counter;
* PgStat_TableCounts The actual per-table counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
- * It is a component of PgStat_TableStatus (within-backend state) and
- * PgStat_TableEntry (the transmitted message format).
+ * it against zeroes to detect whether there are any counts to write.
+ * It is a component of PgStat_TableStatus (within-backend state).
*
* Note: for a table, tuples_returned is the number of tuples successfully
* fetched by heap_getnext, while tuples_fetched is the number of tuples
@@ -159,10 +129,13 @@ typedef enum PgStat_Single_Reset_Type
*/
typedef struct PgStat_TableStatus
{
- Oid t_id; /* table's OID */
- bool t_shared; /* is it a shared catalog? */
struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_TableCounts t_counts; /* event counts to be sent */
+ Relation relation; /* rel that is using this entry */
} PgStat_TableStatus;
/* ----------
@@ -186,353 +159,57 @@ typedef struct PgStat_TableXactStatus
struct PgStat_TableXactStatus *next; /* next of same subxact */
} PgStat_TableXactStatus;
-
-/* ------------------------------------------------------------
- * Message formats follow
- * ------------------------------------------------------------
- */
-
-
-/* ----------
- * PgStat_MsgHdr The common message header
- * ----------
+/*
+ * Archiver statistics kept in the shared stats
*/
-typedef struct PgStat_MsgHdr
+typedef struct PgStat_Archiver
{
- StatMsgType m_type;
- int m_size;
-} PgStat_MsgHdr;
-
-/* ----------
- * Space available in a message. This will keep the UDP packets below 1K,
- * which should fit unfragmented into the MTU of the loopback interface.
- * (Larger values of PGSTAT_MAX_MSG_SIZE would work for that on most
- * platforms, but we're being conservative here.)
- * ----------
- */
-#define PGSTAT_MAX_MSG_SIZE 1000
-#define PGSTAT_MSG_PAYLOAD (PGSTAT_MAX_MSG_SIZE - sizeof(PgStat_MsgHdr))
-
+ PgStat_Counter archived_count; /* archival successes */
+ char last_archived_wal[MAX_XFN_CHARS + 1]; /* last WAL file
+ * archived */
+ TimestampTz last_archived_timestamp; /* last archival success time */
+ PgStat_Counter failed_count; /* failed archival attempts */
+ char last_failed_wal[MAX_XFN_CHARS + 1]; /* WAL file involved in
+ * last failure */
+ TimestampTz last_failed_timestamp; /* last archival failure time */
+ TimestampTz stat_reset_timestamp;
+} PgStat_Archiver;
/* ----------
- * PgStat_MsgDummy A dummy message, ignored by the collector
+ * PgStat_BgWriter bgwriter statistics
* ----------
*/
-typedef struct PgStat_MsgDummy
+typedef struct PgStat_BgWriter
{
- PgStat_MsgHdr m_hdr;
-} PgStat_MsgDummy;
-
+ PgStat_Counter buf_written_clean;
+ PgStat_Counter maxwritten_clean;
+ PgStat_Counter buf_alloc;
+ TimestampTz stat_reset_timestamp;
+} PgStat_BgWriter;
/* ----------
- * PgStat_MsgInquiry Sent by a backend to ask the collector
- * to write the stats file(s).
- *
- * Ordinarily, an inquiry message prompts writing of the global stats file,
- * the stats file for shared catalogs, and the stats file for the specified
- * database. If databaseid is InvalidOid, only the first two are written.
- *
- * New file(s) will be written only if the existing file has a timestamp
- * older than the specified cutoff_time; this prevents duplicated effort
- * when multiple requests arrive at nearly the same time, assuming that
- * backends send requests with cutoff_times a little bit in the past.
- *
- * clock_time should be the requestor's current local time; the collector
- * uses this to check for the system clock going backward, but it has no
- * effect unless that occurs. We assume clock_time >= cutoff_time, though.
+ * PgStat_CheckPointer checkpointer statistics
* ----------
*/
-
-typedef struct PgStat_MsgInquiry
+typedef struct PgStat_CheckPointer
{
- PgStat_MsgHdr m_hdr;
- TimestampTz clock_time; /* observed local clock time */
- TimestampTz cutoff_time; /* minimum acceptable file timestamp */
- Oid databaseid; /* requested DB (InvalidOid => shared only) */
-} PgStat_MsgInquiry;
-
-
-/* ----------
- * PgStat_TableEntry Per-table info in a MsgTabstat
- * ----------
- */
-typedef struct PgStat_TableEntry
-{
- Oid t_id;
- PgStat_TableCounts t_counts;
-} PgStat_TableEntry;
-
-/* ----------
- * PgStat_MsgTabstat Sent by the backend to report table
- * and buffer access statistics.
- * ----------
- */
-#define PGSTAT_NUM_TABENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - 3 * sizeof(int) - 2 * sizeof(PgStat_Counter)) \
- / sizeof(PgStat_TableEntry))
-
-typedef struct PgStat_MsgTabstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- int m_xact_commit;
- int m_xact_rollback;
- PgStat_Counter m_block_read_time; /* times in microseconds */
- PgStat_Counter m_block_write_time;
- PgStat_TableEntry m_entry[PGSTAT_NUM_TABENTRIES];
-} PgStat_MsgTabstat;
-
-
-/* ----------
- * PgStat_MsgTabpurge Sent by the backend to tell the collector
- * about dead tables.
- * ----------
- */
-#define PGSTAT_NUM_TABPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgTabpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_tableid[PGSTAT_NUM_TABPURGE];
-} PgStat_MsgTabpurge;
-
-
-/* ----------
- * PgStat_MsgDropdb Sent by the backend to tell the collector
- * about a dropped database
- * ----------
- */
-typedef struct PgStat_MsgDropdb
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDropdb;
-
-
-/* ----------
- * PgStat_MsgResetcounter Sent by the backend to tell the collector
- * to reset counters
- * ----------
- */
-typedef struct PgStat_MsgResetcounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgResetcounter;
-
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
- */
-typedef struct PgStat_MsgResetsharedcounter
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Shared_Reset_Target m_resettarget;
-} PgStat_MsgResetsharedcounter;
-
-/* ----------
- * PgStat_MsgResetsinglecounter Sent by the backend to tell the collector
- * to reset a single counter
- * ----------
- */
-typedef struct PgStat_MsgResetsinglecounter
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- PgStat_Single_Reset_Type m_resettype;
- Oid m_objectid;
-} PgStat_MsgResetsinglecounter;
-
-/* ----------
- * PgStat_MsgResetslrucounter Sent by the backend to tell the collector
- * to reset a SLRU counter
- * ----------
- */
-typedef struct PgStat_MsgResetslrucounter
-{
- PgStat_MsgHdr m_hdr;
- int m_index;
-} PgStat_MsgResetslrucounter;
-
-/* ----------
- * PgStat_MsgResetreplslotcounter Sent by the backend to tell the collector
- * to reset replication slot counter(s)
- * ----------
- */
-typedef struct PgStat_MsgResetreplslotcounter
-{
- PgStat_MsgHdr m_hdr;
- char m_slotname[NAMEDATALEN];
- bool clearall;
-} PgStat_MsgResetreplslotcounter;
-
-/* ----------
- * PgStat_MsgAutovacStart Sent by the autovacuum daemon to signal
- * that a database is going to be processed
- * ----------
- */
-typedef struct PgStat_MsgAutovacStart
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- TimestampTz m_start_time;
-} PgStat_MsgAutovacStart;
-
-
-/* ----------
- * PgStat_MsgVacuum Sent by the backend or autovacuum daemon
- * after VACUUM
- * ----------
- */
-typedef struct PgStat_MsgVacuum
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- TimestampTz m_vacuumtime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgVacuum;
-
-
-/* ----------
- * PgStat_MsgAnalyze Sent by the backend or autovacuum daemon
- * after ANALYZE
- * ----------
- */
-typedef struct PgStat_MsgAnalyze
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- Oid m_tableoid;
- bool m_autovacuum;
- bool m_resetcounter;
- TimestampTz m_analyzetime;
- PgStat_Counter m_live_tuples;
- PgStat_Counter m_dead_tuples;
-} PgStat_MsgAnalyze;
-
-
-/* ----------
- * PgStat_MsgArchiver Sent by the archiver to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgArchiver
-{
- PgStat_MsgHdr m_hdr;
- bool m_failed; /* Failed attempt */
- char m_xlog[MAX_XFN_CHARS + 1];
- TimestampTz m_timestamp;
-} PgStat_MsgArchiver;
-
-/* ----------
- * PgStat_MsgBgWriter Sent by the bgwriter to update statistics.
- * ----------
- */
-typedef struct PgStat_MsgBgWriter
-{
- PgStat_MsgHdr m_hdr;
-
- PgStat_Counter m_timed_checkpoints;
- PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_clean;
- PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
- PgStat_Counter m_buf_alloc;
- PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter m_checkpoint_sync_time;
-} PgStat_MsgBgWriter;
-
-/* ----------
- * PgStat_MsgWal Sent by backends and background processes to update WAL statistics.
- * ----------
- */
-typedef struct PgStat_MsgWal
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_wal_records;
- PgStat_Counter m_wal_fpi;
- uint64 m_wal_bytes;
- PgStat_Counter m_wal_buffers_full;
-} PgStat_MsgWal;
-
-/* ----------
- * PgStat_MsgSLRU Sent by a backend to update SLRU statistics.
- * ----------
- */
-typedef struct PgStat_MsgSLRU
-{
- PgStat_MsgHdr m_hdr;
- PgStat_Counter m_index;
- PgStat_Counter m_blocks_zeroed;
- PgStat_Counter m_blocks_hit;
- PgStat_Counter m_blocks_read;
- PgStat_Counter m_blocks_written;
- PgStat_Counter m_blocks_exists;
- PgStat_Counter m_flush;
- PgStat_Counter m_truncate;
-} PgStat_MsgSLRU;
-
-/* ----------
- * PgStat_MsgReplSlot Sent by a backend or a wal sender to update replication
- * slot statistics.
- * ----------
- */
-typedef struct PgStat_MsgReplSlot
-{
- PgStat_MsgHdr m_hdr;
- char m_slotname[NAMEDATALEN];
- bool m_drop;
- PgStat_Counter m_spill_txns;
- PgStat_Counter m_spill_count;
- PgStat_Counter m_spill_bytes;
- PgStat_Counter m_stream_txns;
- PgStat_Counter m_stream_count;
- PgStat_Counter m_stream_bytes;
-} PgStat_MsgReplSlot;
-
-
-/* ----------
- * PgStat_MsgRecoveryConflict Sent by the backend upon recovery conflict
- * ----------
- */
-typedef struct PgStat_MsgRecoveryConflict
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- int m_reason;
-} PgStat_MsgRecoveryConflict;
-
-/* ----------
- * PgStat_MsgTempFile Sent by the backend upon creating a temp file
- * ----------
- */
-typedef struct PgStat_MsgTempFile
-{
- PgStat_MsgHdr m_hdr;
-
- Oid m_databaseid;
- size_t m_filesize;
-} PgStat_MsgTempFile;
+ PgStat_Counter timed_checkpoints;
+ PgStat_Counter requested_checkpoints;
+ PgStat_Counter buf_written_checkpoints;
+ PgStat_Counter buf_written_backend;
+ PgStat_Counter buf_fsync_backend;
+ PgStat_Counter checkpoint_write_time; /* times in milliseconds */
+ PgStat_Counter checkpoint_sync_time;
+} PgStat_CheckPointer;
/* ----------
* PgStat_FunctionCounts The actual per-function counts kept by a backend
*
* This struct should contain only actual event counters, because we memcmp
- * it against zeroes to detect whether there are any counts to transmit.
+ * it against zeroes to detect whether there are any counts to write.
*
* Note that the time counters are in instr_time format here. We convert to
- * microseconds in PgStat_Counter format when transmitting to the collector.
+ * microseconds in PgStat_Counter format when writing to shared statsitics.
* ----------
*/
typedef struct PgStat_FunctionCounts
@@ -548,7 +225,6 @@ typedef struct PgStat_FunctionCounts
*/
typedef struct PgStat_BackendFunctionEntry
{
- Oid f_id;
PgStat_FunctionCounts f_counts;
} PgStat_BackendFunctionEntry;
@@ -564,101 +240,8 @@ typedef struct PgStat_FunctionEntry
PgStat_Counter f_self_time;
} PgStat_FunctionEntry;
-/* ----------
- * PgStat_MsgFuncstat Sent by the backend to report function
- * usage statistics.
- * ----------
- */
-#define PGSTAT_NUM_FUNCENTRIES \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(PgStat_FunctionEntry))
-
-typedef struct PgStat_MsgFuncstat
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- PgStat_FunctionEntry m_entry[PGSTAT_NUM_FUNCENTRIES];
-} PgStat_MsgFuncstat;
-
-/* ----------
- * PgStat_MsgFuncpurge Sent by the backend to tell the collector
- * about dead functions.
- * ----------
- */
-#define PGSTAT_NUM_FUNCPURGE \
- ((PGSTAT_MSG_PAYLOAD - sizeof(Oid) - sizeof(int)) \
- / sizeof(Oid))
-
-typedef struct PgStat_MsgFuncpurge
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_nentries;
- Oid m_functionid[PGSTAT_NUM_FUNCPURGE];
-} PgStat_MsgFuncpurge;
-
-/* ----------
- * PgStat_MsgDeadlock Sent by the backend to tell the collector
- * about a deadlock that occurred.
- * ----------
- */
-typedef struct PgStat_MsgDeadlock
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
-} PgStat_MsgDeadlock;
-
-/* ----------
- * PgStat_MsgChecksumFailure Sent by the backend to tell the collector
- * about checksum failures noticed.
- * ----------
- */
-typedef struct PgStat_MsgChecksumFailure
-{
- PgStat_MsgHdr m_hdr;
- Oid m_databaseid;
- int m_failurecount;
- TimestampTz m_failure_time;
-} PgStat_MsgChecksumFailure;
-
-
-/* ----------
- * PgStat_Msg Union over all possible messages.
- * ----------
- */
-typedef union PgStat_Msg
-{
- PgStat_MsgHdr msg_hdr;
- PgStat_MsgDummy msg_dummy;
- PgStat_MsgInquiry msg_inquiry;
- PgStat_MsgTabstat msg_tabstat;
- PgStat_MsgTabpurge msg_tabpurge;
- PgStat_MsgDropdb msg_dropdb;
- PgStat_MsgResetcounter msg_resetcounter;
- PgStat_MsgResetsharedcounter msg_resetsharedcounter;
- PgStat_MsgResetsinglecounter msg_resetsinglecounter;
- PgStat_MsgResetslrucounter msg_resetslrucounter;
- PgStat_MsgResetreplslotcounter msg_resetreplslotcounter;
- PgStat_MsgAutovacStart msg_autovacuum_start;
- PgStat_MsgVacuum msg_vacuum;
- PgStat_MsgAnalyze msg_analyze;
- PgStat_MsgArchiver msg_archiver;
- PgStat_MsgBgWriter msg_bgwriter;
- PgStat_MsgWal msg_wal;
- PgStat_MsgSLRU msg_slru;
- PgStat_MsgFuncstat msg_funcstat;
- PgStat_MsgFuncpurge msg_funcpurge;
- PgStat_MsgRecoveryConflict msg_recoveryconflict;
- PgStat_MsgDeadlock msg_deadlock;
- PgStat_MsgTempFile msg_tempfile;
- PgStat_MsgChecksumFailure msg_checksumfailure;
- PgStat_MsgReplSlot msg_replslot;
-} PgStat_Msg;
-
-
/* ------------------------------------------------------------
- * Statistic collector data structures follow
+ * Activity statistics data structures on file and shared memory follow
*
* PGSTAT_FILE_FORMAT_ID should be changed whenever any of these
* data structures change.
@@ -667,13 +250,9 @@ typedef union PgStat_Msg
#define PGSTAT_FILE_FORMAT_ID 0x01A5BC9F
-/* ----------
- * PgStat_StatDBEntry The collector's data per database
- * ----------
- */
-typedef struct PgStat_StatDBEntry
+
+typedef struct PgStat_StatDBCounts
{
- Oid databaseid;
PgStat_Counter n_xact_commit;
PgStat_Counter n_xact_rollback;
PgStat_Counter n_blocks_fetched;
@@ -683,7 +262,6 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_tuples_inserted;
PgStat_Counter n_tuples_updated;
PgStat_Counter n_tuples_deleted;
- TimestampTz last_autovac_time;
PgStat_Counter n_conflict_tablespace;
PgStat_Counter n_conflict_lock;
PgStat_Counter n_conflict_snapshot;
@@ -693,29 +271,86 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter n_temp_bytes;
PgStat_Counter n_deadlocks;
PgStat_Counter n_checksum_failures;
- TimestampTz last_checksum_failure;
PgStat_Counter n_block_read_time; /* times in microseconds */
PgStat_Counter n_block_write_time;
+} PgStat_StatDBCounts;
+/* ----------
+ * PgStat_StatEntryHead common header struct for PgStat_Stat*Entry
+ * ----------
+ */
+typedef struct PgStat_StatEntryHeader
+{
+ LWLock lock;
+ bool dropped; /* This entry is being dropped and should
+ * be removed when refcount goes to
+ * zero. */
+ pg_atomic_uint32 refcount; /* How many backends are referenceing */
+} PgStat_StatEntryHeader;
+
+/* ----------
+ * PgStat_StatDBEntry The statistics per database
+ * ----------
+ */
+typedef struct PgStat_StatDBEntry
+{
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ TimestampTz last_autovac_time;
+ TimestampTz last_checksum_failure;
TimestampTz stat_reset_timestamp;
- TimestampTz stats_timestamp; /* time of db stats file update */
+ TimestampTz stats_timestamp; /* time of db stats update */
+
+ PgStat_StatDBCounts counts;
/*
- * tables and functions must be last in the struct, because we don't write
- * the pointers out to the stats file.
+ * The followings must be last in the struct, because we don't write them
+ * out to the stats file.
*/
- HTAB *tables;
- HTAB *functions;
+ dshash_table_handle tables; /* current gen tables hash */
+ dshash_table_handle functions; /* current gen functions hash */
} PgStat_StatDBEntry;
+/* ----------
+ * PgStat_Wal Sent by backends and background processes to
+ * update WAL statistics.
+ * ----------
+ */
+typedef struct PgStat_Wal
+{
+ WalUsage wal_usage;
+ PgStat_Counter wal_buffers_full;
+ TimestampTz stat_reset_timestamp;
+} PgStat_Wal;
+
+/*
+ * SLRU statistics kept in the stats collector
+ */
+typedef struct PgStat_SLRUStats
+{
+ PgStat_Counter blocks_zeroed;
+ PgStat_Counter blocks_hit;
+ PgStat_Counter blocks_read;
+ PgStat_Counter blocks_written;
+ PgStat_Counter blocks_exists;
+ PgStat_Counter flush;
+ PgStat_Counter truncate;
+ TimestampTz stat_reset_timestamp;
+} PgStat_SLRUStats;
/* ----------
- * PgStat_StatTabEntry The collector's data per table (or index)
+ * PgStat_StatTabEntry The statistics per table (or index)
* ----------
*/
typedef struct PgStat_StatTabEntry
{
- Oid tableid;
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ /* Persistent data follow */
+ TimestampTz vacuum_timestamp; /* user initiated vacuum */
+ TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
+ TimestampTz analyze_timestamp; /* user initiated */
+ TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter numscans;
@@ -735,99 +370,35 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
- TimestampTz vacuum_timestamp; /* user initiated vacuum */
PgStat_Counter vacuum_count;
- TimestampTz autovac_vacuum_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_vacuum_count;
- TimestampTz analyze_timestamp; /* user initiated */
PgStat_Counter analyze_count;
- TimestampTz autovac_analyze_timestamp; /* autovacuum initiated */
PgStat_Counter autovac_analyze_count;
} PgStat_StatTabEntry;
/* ----------
- * PgStat_StatFuncEntry The collector's data per function
+ * PgStat_StatFuncEntry per function stats data
* ----------
*/
typedef struct PgStat_StatFuncEntry
{
- Oid functionid;
-
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
+ /* Persistent data follow */
PgStat_Counter f_numcalls;
PgStat_Counter f_total_time; /* times in microseconds */
PgStat_Counter f_self_time;
} PgStat_StatFuncEntry;
-
-/*
- * Archiver statistics kept in the stats collector
- */
-typedef struct PgStat_ArchiverStats
-{
- PgStat_Counter archived_count; /* archival successes */
- char last_archived_wal[MAX_XFN_CHARS + 1]; /* last WAL file
- * archived */
- TimestampTz last_archived_timestamp; /* last archival success time */
- PgStat_Counter failed_count; /* failed archival attempts */
- char last_failed_wal[MAX_XFN_CHARS + 1]; /* WAL file involved in
- * last failure */
- TimestampTz last_failed_timestamp; /* last archival failure time */
- TimestampTz stat_reset_timestamp;
-} PgStat_ArchiverStats;
-
-/*
- * Global statistics kept in the stats collector
- */
-typedef struct PgStat_GlobalStats
-{
- TimestampTz stats_timestamp; /* time of stats file update */
- PgStat_Counter timed_checkpoints;
- PgStat_Counter requested_checkpoints;
- PgStat_Counter checkpoint_write_time; /* times in milliseconds */
- PgStat_Counter checkpoint_sync_time;
- PgStat_Counter buf_written_checkpoints;
- PgStat_Counter buf_written_clean;
- PgStat_Counter maxwritten_clean;
- PgStat_Counter buf_written_backend;
- PgStat_Counter buf_fsync_backend;
- PgStat_Counter buf_alloc;
- TimestampTz stat_reset_timestamp;
-} PgStat_GlobalStats;
-
-/*
- * WAL statistics kept in the stats collector
- */
-typedef struct PgStat_WalStats
-{
- PgStat_Counter wal_records;
- PgStat_Counter wal_fpi;
- uint64 wal_bytes;
- PgStat_Counter wal_buffers_full;
- TimestampTz stat_reset_timestamp;
-} PgStat_WalStats;
-
-/*
- * SLRU statistics kept in the stats collector
- */
-typedef struct PgStat_SLRUStats
-{
- PgStat_Counter blocks_zeroed;
- PgStat_Counter blocks_hit;
- PgStat_Counter blocks_read;
- PgStat_Counter blocks_written;
- PgStat_Counter blocks_exists;
- PgStat_Counter flush;
- PgStat_Counter truncate;
- TimestampTz stat_reset_timestamp;
-} PgStat_SLRUStats;
-
/*
* Replication slot statistics kept in the stats collector
*/
-typedef struct PgStat_ReplSlotStats
+typedef struct PgStat_ReplSlot
{
+ PgStat_StatEntryHeader header; /* must be the first member,
+ used only on shared memory */
char slotname[NAMEDATALEN];
PgStat_Counter spill_txns;
PgStat_Counter spill_count;
@@ -836,7 +407,7 @@ typedef struct PgStat_ReplSlotStats
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
TimestampTz stat_reset_timestamp;
-} PgStat_ReplSlotStats;
+} PgStat_ReplSlot;
/* ----------
* Backend states
@@ -885,7 +456,7 @@ typedef enum
WAIT_EVENT_CHECKPOINTER_MAIN,
WAIT_EVENT_LOGICAL_APPLY_MAIN,
WAIT_EVENT_LOGICAL_LAUNCHER_MAIN,
- WAIT_EVENT_PGSTAT_MAIN,
+ WAIT_EVENT_READING_STATS_FILE,
WAIT_EVENT_RECOVERY_WAL_STREAM,
WAIT_EVENT_SYSLOGGER_MAIN,
WAIT_EVENT_WAL_RECEIVER_MAIN,
@@ -1138,7 +709,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that the collector process
+ * BackendId, but that is not critical.) Note that the stats-writing functions
* has no involvement in, or even access to, these structs.
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -1335,18 +906,26 @@ extern PGDLLIMPORT bool pgstat_track_counts;
extern PGDLLIMPORT int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
extern char *pgstat_stat_directory;
+
+/* No longer used, but will be removed with GUC */
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
/*
* BgWriter statistics counters are updated directly by bgwriter and bufmgr
*/
-extern PgStat_MsgBgWriter BgWriterStats;
+extern PgStat_BgWriter BgWriterStats;
+
+/*
+ * CheckPointer statistics counters are updated directly by checkpointer and
+ * bufmgr
+ */
+extern PgStat_CheckPointer CheckPointerStats;
/*
* WAL statistics counter is updated by backends and background processes
*/
-extern PgStat_MsgWal WalStats;
+extern PgStat_Wal WalStats;
/*
* Updated by pgstat_count_buffer_*_time macros
@@ -1361,33 +940,27 @@ extern PgStat_Counter pgStatBlockWriteTime;
extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
-extern void pgstat_init(void);
-extern int pgstat_start(void);
+extern Size StatsShmemSize(void);
+extern void StatsShmemInit(void);
+
extern void pgstat_reset_all(void);
-extern void allow_immediate_pgstat_restart(void);
-
-#ifdef EXEC_BACKEND
-extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
* ----------
*/
-extern void pgstat_ping(void);
-
-extern void pgstat_report_stat(bool force);
+extern long pgstat_report_stat(bool force);
extern void pgstat_vacuum_stat(void);
extern void pgstat_drop_database(Oid databaseid);
extern void pgstat_clear_snapshot(void);
extern void pgstat_reset_counters(void);
-extern void pgstat_reset_shared_counters(const char *);
+extern void pgstat_reset_shared_counters(const char *target);
extern void pgstat_reset_single_counter(Oid objectid, PgStat_Single_Reset_Type type);
extern void pgstat_reset_slru_counter(const char *);
extern void pgstat_reset_replslot_counter(const char *name);
+extern void pgstat_report_wal(void);
extern void pgstat_report_autovac(Oid dboid);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
PgStat_Counter livetuples, PgStat_Counter deadtuples);
@@ -1427,6 +1000,7 @@ extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
extern void pgstat_initstats(Relation rel);
+extern void pgstat_delinkstats(Relation rel);
extern char *pgstat_clip_activity(const char *raw_activity);
@@ -1549,9 +1123,10 @@ extern void pgstat_twophase_postcommit(TransactionId xid, uint16 info,
extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
void *recdata, uint32 len);
-extern void pgstat_send_archiver(const char *xlog, bool failed);
-extern void pgstat_send_bgwriter(void);
-extern void pgstat_send_wal(void);
+extern void pgstat_report_archiver(const char *xlog, bool failed);
+extern void pgstat_report_bgwriter(void);
+extern void pgstat_report_checkpointer(void);
+extern void pgstat_report_wal(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -1560,15 +1135,20 @@ extern void pgstat_send_wal(void);
*/
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
+extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_extended(bool shared,
+ Oid relid);
+extern void pgstat_copy_index_counters(Oid relid, PgStat_TableStatus *dst);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
-extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
-extern PgStat_GlobalStats *pgstat_fetch_global(void);
-extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
+extern TimestampTz pgstat_get_stat_timestamp(void);
+extern PgStat_Archiver *pgstat_fetch_stat_archiver(void);
+extern PgStat_BgWriter *pgstat_fetch_stat_bgwriter(void);
+extern PgStat_CheckPointer *pgstat_fetch_stat_checkpointer(void);
+extern PgStat_Wal *pgstat_fetch_stat_wal(void);
extern PgStat_SLRUStats *pgstat_fetch_slru(void);
-extern PgStat_ReplSlotStats *pgstat_fetch_replslot(int *nslots_p);
+extern PgStat_ReplSlot *pgstat_fetch_replslot(int *nslots_p);
extern void pgstat_count_slru_page_zeroed(int slru_idx);
extern void pgstat_count_slru_page_hit(int slru_idx);
@@ -1579,5 +1159,6 @@ extern void pgstat_count_slru_flush(int slru_idx);
extern void pgstat_count_slru_truncate(int slru_idx);
extern const char *pgstat_slru_name(int slru_idx);
extern int pgstat_slru_index(const char *name);
+extern void pgstat_clear_snapshot(void);
#endif /* PGSTAT_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index cbf2510fbf..9ed6b54428 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -218,6 +218,7 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_SHARED_TIDBITMAP,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_PER_XACT_PREDICATE_LIST,
+ LWTRANCHE_STATS,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index b9b5c1adda..add9c53ee3 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -88,7 +88,7 @@ enum config_group
PROCESS_TITLE,
STATS,
STATS_MONITORING,
- STATS_COLLECTOR,
+ STATS_ACTIVITY,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index ecb2a366a5..f090f7372a 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -32,6 +32,7 @@ typedef enum TimeoutId
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IDLE_SESSION_TIMEOUT,
+ IDLE_STATS_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
--
2.27.0
----Next_Part(Fri_Jan__8_10_24_34_2021_185)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v45-0005-Doc-part-of-shared-memory-based-stats-collector.patch"
^ permalink raw reply [nested|flat] 124+ messages in thread
* RFC: Logging plan of the running query
@ 2021-05-12 11:24 torikoshia <[email protected]>
2021-05-12 12:07 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-05-12 12:33 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-12 12:55 ` Re: RFC: Logging plan of the running query Matthias van de Meent <[email protected]>
2021-05-12 14:40 ` Re: RFC: Logging plan of the running query Julien Rouhaud <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
0 siblings, 5 replies; 124+ messages in thread
From: torikoshia @ 2021-05-12 11:24 UTC (permalink / raw)
To: pgsql-hackers
Hi,
During the discussion about memory contexts dumping[1], there
was a comment that exposing not only memory contexts but also
query plans and untruncated query string would be useful.
I also feel that it would be nice when thinking about situations
such as troubleshooting a long-running query on production
environments where we cannot use debuggers.
At that point of the above comment, I was considering exposing
such information on the shared memory.
However, since memory contexts are now exposed on the log by
pg_log_backend_memory_contexts(PID), I'm thinking about
defining a function that logs the plan of a running query and
untruncated query string on the specified PID in the same way
as below.
postgres=# SELECT * FROM pg_log_current_plan(2155192);
pg_log_current_plan
---------------------
t
(1 row)
$ tail -f data/log/postgresql-2021-05-12.log
2021-05-12 17:37:19.481 JST [2155192] LOG: logging the plan of
running query on PID 2155192
Query Text: SELECT a.filler FROM pgbench_accounts a JOIN
pgbench_accounts b ON a.aid = b.aid;
Merge Join (cost=0.85..83357.85 rows=1000000 width=85)
Merge Cond: (a.aid = b.aid)
-> Index Scan using pgbench_accounts_pkey on
pgbench_accounts a (cost=0.42..42377.43 rows=1000000 width=89)
-> Index Only Scan using pgbench_accounts_pkey on
pgbench_accounts b (cost=0.42..25980.42 rows=1000000 width=4)
Attached a PoC patch.
Any thoughts?
[1]
https://www.postgresql.org/message-id/CA%2BTgmobkpFV0UB67kzXuD36--OFHwz1bs%3DL_6PZbD4nxKqUQMw%40mail...
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v1-0001-log-running-query-plan.patch (9.8K, ../../[email protected]/2-v1-0001-log-running-query-plan.patch)
download | inline diff:
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4d1f1794ca..5959bbfddc 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24939,6 +24939,26 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_plan</primary>
+ </indexterm>
+ <function>pg_log_current_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the untruncated query string and its plan of the
+ backend with the specified process ID.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log the plan of running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 9867da83bc..edbe1c6829 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -4928,3 +4931,116 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the
+ * plan of a running query.
+ *
+ * All the actual work is deferred to ProcessLogExplainInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange
+ * to call this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because
+ * the target process for logging the plan of running query.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es = NewExplainState();
+
+ LogCurrentPlanPending = false;
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->summary = true;
+
+ if (ActivePortal && ActivePortal->queryDesc != NULL)
+ {
+ ExplainQueryText(es, ActivePortal->queryDesc);
+ ExplainPrintPlan(es, ActivePortal->queryDesc);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG,
+ (errmsg("logging the plan of running query on PID %d\n%s",
+ MyProcPid,
+ es->str->data),
+ errhidestmt(true)));
+ }
+ else
+ ereport(LOG,
+ (errmsg("PID %d is not running a query now",
+ MyProcPid)));
+}
+
+/*
+ * pg_log_current_plan
+ * Signal a backend process to log the plan of running
+ * query.
+ *
+ * Only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial
+ * of service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * plan.
+ */
+Datum
+pg_log_current_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to look into the plan of long running query, that it might end on
+ * its own first and its plan is not logged is not a problem.
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ /* Only allow superusers to log the plan of running query. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log the plan of running query")));
+
+ if (SendProcSignal(pid, PROCSIG_LOG_CURRENT_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index eac6895141..3aaf4b4818 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 2d6d145ecc..f456f0b7d5 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3356,6 +3357,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 26c3fc0f6b..461c1efe91 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7975,6 +7975,12 @@
provolatile => 'v', prorettype => 'bool',
proargtypes => 'int4', prosrc => 'pg_log_backend_memory_contexts' },
+# logging the plan of running query on the specified backend
+{ oid => '4544', descr => 'log the plan of running query on the specified backend',
+ proname => 'pg_log_current_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..fa81beb553 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -124,4 +124,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 95202d37af..8c03a515fd 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -85,6 +85,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..a80e7999e8 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log the plan of current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-05-12 12:07 ` Pavel Stehule <[email protected]>
4 siblings, 0 replies; 124+ messages in thread
From: Pavel Stehule @ 2021-05-12 12:07 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: pgsql-hackers
st 12. 5. 2021 v 13:24 odesílatel torikoshia <[email protected]>
napsal:
> Hi,
>
> During the discussion about memory contexts dumping[1], there
> was a comment that exposing not only memory contexts but also
> query plans and untruncated query string would be useful.
>
> I also feel that it would be nice when thinking about situations
> such as troubleshooting a long-running query on production
> environments where we cannot use debuggers.
>
> At that point of the above comment, I was considering exposing
> such information on the shared memory.
> However, since memory contexts are now exposed on the log by
> pg_log_backend_memory_contexts(PID), I'm thinking about
> defining a function that logs the plan of a running query and
> untruncated query string on the specified PID in the same way
> as below.
>
> postgres=# SELECT * FROM pg_log_current_plan(2155192);
> pg_log_current_plan
> ---------------------
> t
> (1 row)
>
> $ tail -f data/log/postgresql-2021-05-12.log
>
> 2021-05-12 17:37:19.481 JST [2155192] LOG: logging the plan of
> running query on PID 2155192
> Query Text: SELECT a.filler FROM pgbench_accounts a JOIN
> pgbench_accounts b ON a.aid = b.aid;
> Merge Join (cost=0.85..83357.85 rows=1000000 width=85)
> Merge Cond: (a.aid = b.aid)
> -> Index Scan using pgbench_accounts_pkey on
> pgbench_accounts a (cost=0.42..42377.43 rows=1000000 width=89)
> -> Index Only Scan using pgbench_accounts_pkey on
> pgbench_accounts b (cost=0.42..25980.42 rows=1000000 width=4)
>
>
> Attached a PoC patch.
>
> Any thoughts?
>
+1
Pavel
> [1]
>
> https://www.postgresql.org/message-id/CA%2BTgmobkpFV0UB67kzXuD36--OFHwz1bs%3DL_6PZbD4nxKqUQMw%40mail...
>
>
> Regards,
>
> --
> Atsushi Torikoshi
> NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-05-12 12:33 ` Bharath Rupireddy <[email protected]>
2021-05-12 16:08 ` Re: RFC: Logging plan of the running query Laurenz Albe <[email protected]>
4 siblings, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-05-12 12:33 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: pgsql-hackers
On Wed, May 12, 2021 at 4:54 PM torikoshia <[email protected]> wrote:
>
> Hi,
>
> During the discussion about memory contexts dumping[1], there
> was a comment that exposing not only memory contexts but also
> query plans and untruncated query string would be useful.
>
> I also feel that it would be nice when thinking about situations
> such as troubleshooting a long-running query on production
> environments where we cannot use debuggers.
>
> At that point of the above comment, I was considering exposing
> such information on the shared memory.
> However, since memory contexts are now exposed on the log by
> pg_log_backend_memory_contexts(PID), I'm thinking about
> defining a function that logs the plan of a running query and
> untruncated query string on the specified PID in the same way
> as below.
>
> postgres=# SELECT * FROM pg_log_current_plan(2155192);
> pg_log_current_plan
> ---------------------
> t
> (1 row)
>
> $ tail -f data/log/postgresql-2021-05-12.log
>
> 2021-05-12 17:37:19.481 JST [2155192] LOG: logging the plan of
> running query on PID 2155192
> Query Text: SELECT a.filler FROM pgbench_accounts a JOIN
> pgbench_accounts b ON a.aid = b.aid;
> Merge Join (cost=0.85..83357.85 rows=1000000 width=85)
> Merge Cond: (a.aid = b.aid)
> -> Index Scan using pgbench_accounts_pkey on
> pgbench_accounts a (cost=0.42..42377.43 rows=1000000 width=89)
> -> Index Only Scan using pgbench_accounts_pkey on
> pgbench_accounts b (cost=0.42..25980.42 rows=1000000 width=4)
>
>
> Attached a PoC patch.
>
> Any thoughts?
>
> [1]
> https://www.postgresql.org/message-id/CA%2BTgmobkpFV0UB67kzXuD36--OFHwz1bs%3DL_6PZbD4nxKqUQMw%40mail...
+1 for the idea. It looks like pg_log_current_plan is allowed to run
by superusers. Since it also shows up the full query text and the plan
in the server log as plain text, there are chances that the sensitive
information might be logged into the server log which is a risky thing
from security standpoint. There's another thread (see [1] below) which
discusses this issue by having a separate role for all debugging
purposes. Note that final consensus is not reached yet. We may want to
use the same role for this patch as well.
[1] - https://www.postgresql.org/message-id/CA%2BTgmoZz%3DK1bQRp0Ug%3D6uMGFWg-6kaxdHe6VSWaxq0U-YkppYQ%40ma...
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-12 12:33 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-05-12 16:08 ` Laurenz Albe <[email protected]>
2021-05-13 08:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Laurenz Albe @ 2021-05-12 16:08 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; torikoshia <[email protected]>; +Cc: pgsql-hackers
On Wed, 2021-05-12 at 18:03 +0530, Bharath Rupireddy wrote:
> On Wed, May 12, 2021 at 4:54 PM torikoshia <[email protected]> wrote:
> > During the discussion about memory contexts dumping[1], there
> > was a comment that exposing not only memory contexts but also
> > query plans and untruncated query string would be useful.
> >
> > postgres=# SELECT * FROM pg_log_current_plan(2155192);
> > pg_log_current_plan
> > ---------------------
> > t
> > (1 row)
> >
> > $ tail -f data/log/postgresql-2021-05-12.log
> >
> > 2021-05-12 17:37:19.481 JST [2155192] LOG: logging the plan of
> > running query on PID 2155192
> > Query Text: SELECT a.filler FROM pgbench_accounts a JOIN
> > pgbench_accounts b ON a.aid = b.aid;
> > Merge Join (cost=0.85..83357.85 rows=1000000 width=85)
> > Merge Cond: (a.aid = b.aid)
> > -> Index Scan using pgbench_accounts_pkey on
> > pgbench_accounts a (cost=0.42..42377.43 rows=1000000 width=89)
> > -> Index Only Scan using pgbench_accounts_pkey on
> > pgbench_accounts b (cost=0.42..25980.42 rows=1000000 width=4)
I love the idea, but I didn't look at the patch.
> Since it also shows up the full query text and the plan
> in the server log as plain text, there are chances that the sensitive
> information might be logged into the server log which is a risky thing
> from security standpoint.
I think that is irrelevant.
A superuser can already set "log_statement = 'all'" to get this.
There is no protection from superusers, and it is pointless to require that.
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-12 12:33 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-12 16:08 ` Re: RFC: Logging plan of the running query Laurenz Albe <[email protected]>
@ 2021-05-13 08:26 ` torikoshia <[email protected]>
2021-05-13 09:38 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-05-13 08:26 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; +Cc: Bharath Rupireddy <[email protected]>; pgsql-hackers
On 2021-05-13 01:08, Laurenz Albe wrote:
> On Wed, 2021-05-12 at 18:03 +0530, Bharath Rupireddy wrote:
>> Since it also shows up the full query text and the plan
>> in the server log as plain text, there are chances that the sensitive
>> information might be logged into the server log which is a risky thing
>> from security standpoint.
Thanks for the notification!
> I think that is irrelevant.
>
> A superuser can already set "log_statement = 'all'" to get this.
> There is no protection from superusers, and it is pointless to require
> that.
AFAIU, since that discussion is whether or not users other than
superusers
should be given the privilege to execute the backtrace printing
function,
I think it might be applicable to pg_log_current_plan().
Since restricting privilege to superusers is stricter, I'm going to
proceed
as it is for now, but depending on the above discussion, it may be
better to
change it.
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-12 12:33 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-12 16:08 ` Re: RFC: Logging plan of the running query Laurenz Albe <[email protected]>
2021-05-13 08:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-05-13 09:38 ` Bharath Rupireddy <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: Bharath Rupireddy @ 2021-05-13 09:38 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Laurenz Albe <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 1:56 PM torikoshia <[email protected]> wrote:
>
> On 2021-05-13 01:08, Laurenz Albe wrote:
> > On Wed, 2021-05-12 at 18:03 +0530, Bharath Rupireddy wrote:
> >> Since it also shows up the full query text and the plan
> >> in the server log as plain text, there are chances that the sensitive
> >> information might be logged into the server log which is a risky thing
> >> from security standpoint.
>
> Thanks for the notification!
>
> > I think that is irrelevant.
> >
> > A superuser can already set "log_statement = 'all'" to get this.
> > There is no protection from superusers, and it is pointless to require
> > that.
>
> AFAIU, since that discussion is whether or not users other than
> superusers
> should be given the privilege to execute the backtrace printing
> function,
> I think it might be applicable to pg_log_current_plan().
>
> Since restricting privilege to superusers is stricter, I'm going to
> proceed
> as it is for now, but depending on the above discussion, it may be
> better to
> change it.
Yeah, we can keep it as superuser-only for now.
Might be unrelated, but just for info - there's another thread
"Granting control of SUSET gucs to non-superusers" at [1] discussing
the new roles.
[1] - https://www.postgresql.org/message-id/F9408A5A-B20B-42D2-9E7F-49CD3D1547BC%40enterprisedb.com
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-05-12 12:55 ` Matthias van de Meent <[email protected]>
2021-05-13 08:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
4 siblings, 1 reply; 124+ messages in thread
From: Matthias van de Meent @ 2021-05-12 12:55 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: pgsql-hackers
On Wed, 12 May 2021 at 13:24, torikoshia <[email protected]> wrote:
>
> Hi,
>
> During the discussion about memory contexts dumping[1], there
> was a comment that exposing not only memory contexts but also
> query plans and untruncated query string would be useful.
>
> I also feel that it would be nice when thinking about situations
> such as troubleshooting a long-running query on production
> environments where we cannot use debuggers.
>
> At that point of the above comment, I was considering exposing
> such information on the shared memory.
> However, since memory contexts are now exposed on the log by
> pg_log_backend_memory_contexts(PID), I'm thinking about
> defining a function that logs the plan of a running query and
> untruncated query string on the specified PID in the same way
> as below.
>
> postgres=# SELECT * FROM pg_log_current_plan(2155192);
> pg_log_current_plan
> ---------------------
> t
> (1 row)
>
> $ tail -f data/log/postgresql-2021-05-12.log
>
> 2021-05-12 17:37:19.481 JST [2155192] LOG: logging the plan of
> running query on PID 2155192
> Query Text: SELECT a.filler FROM pgbench_accounts a JOIN
> pgbench_accounts b ON a.aid = b.aid;
> Merge Join (cost=0.85..83357.85 rows=1000000 width=85)
> Merge Cond: (a.aid = b.aid)
> -> Index Scan using pgbench_accounts_pkey on
> pgbench_accounts a (cost=0.42..42377.43 rows=1000000 width=89)
> -> Index Only Scan using pgbench_accounts_pkey on
> pgbench_accounts b (cost=0.42..25980.42 rows=1000000 width=4)
>
>
> Attached a PoC patch.
>
> Any thoughts?
Great idea. One feature I'd suggest would be adding a 'format' option
as well, if such feature would be feasable.
With regards,
Matthias van de Meent
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-12 12:55 ` Re: RFC: Logging plan of the running query Matthias van de Meent <[email protected]>
@ 2021-05-13 08:23 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2021-05-13 08:23 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; pgsql-hackers
Thank you all for your positive comments.
On 2021-05-12 21:55, Matthias van de Meent wrote:
> Great idea. One feature I'd suggest would be adding a 'format' option
> as well, if such feature would be feasable.
Thanks for the comment!
During the development of pg_log_backend_memory_contexts(), I tried to
make the number of contexts to record configurable by making it GUC
variable or putting it on the shared memory, but the former seemed an
overkill and the latter introduced some ugly behaviors, so we decided
to make it a static number[1].
I think we face the same difficulty here.
Allowing to select the format would be better as auto_explain does by
auto_explain.log_format, but I'm a bit doubtful that it is worth the
costs.
[1]
https://www.postgresql.org/message-id/flat/6738f309-a41b-cbe6-bb57-a1c58ce9f05a%40oss.nttdata.com#e6...
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-05-12 14:40 ` Julien Rouhaud <[email protected]>
4 siblings, 0 replies; 124+ messages in thread
From: Julien Rouhaud @ 2021-05-12 14:40 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: pgsql-hackers
On Wed, May 12, 2021 at 08:24:04PM +0900, torikoshia wrote:
> Hi,
>
> During the discussion about memory contexts dumping[1], there
> was a comment that exposing not only memory contexts but also
> query plans and untruncated query string would be useful.
>
> I also feel that it would be nice when thinking about situations
> such as troubleshooting a long-running query on production
> environments where we cannot use debuggers.
>
> At that point of the above comment, I was considering exposing
> such information on the shared memory.
> However, since memory contexts are now exposed on the log by
> pg_log_backend_memory_contexts(PID), I'm thinking about
> defining a function that logs the plan of a running query and
> untruncated query string on the specified PID in the same way
> as below.
>
> postgres=# SELECT * FROM pg_log_current_plan(2155192);
> pg_log_current_plan
> ---------------------
> t
> (1 row)
>
> $ tail -f data/log/postgresql-2021-05-12.log
>
> 2021-05-12 17:37:19.481 JST [2155192] LOG: logging the plan of running
> query on PID 2155192
> Query Text: SELECT a.filler FROM pgbench_accounts a JOIN
> pgbench_accounts b ON a.aid = b.aid;
> Merge Join (cost=0.85..83357.85 rows=1000000 width=85)
> Merge Cond: (a.aid = b.aid)
> -> Index Scan using pgbench_accounts_pkey on pgbench_accounts a
> (cost=0.42..42377.43 rows=1000000 width=89)
> -> Index Only Scan using pgbench_accounts_pkey on
> pgbench_accounts b (cost=0.42..25980.42 rows=1000000 width=4)
I didn't read the POC patch yet, but +1 for having that feature.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-05-13 09:13 ` Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
4 siblings, 1 reply; 124+ messages in thread
From: Dilip Kumar @ 2021-05-13 09:13 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: pgsql-hackers
On Wed, May 12, 2021 at 4:54 PM torikoshia <[email protected]> wrote:
>
> Hi,
>
> During the discussion about memory contexts dumping[1], there
> was a comment that exposing not only memory contexts but also
> query plans and untruncated query string would be useful.
>
> I also feel that it would be nice when thinking about situations
> such as troubleshooting a long-running query on production
> environments where we cannot use debuggers.
>
> At that point of the above comment, I was considering exposing
> such information on the shared memory.
> However, since memory contexts are now exposed on the log by
> pg_log_backend_memory_contexts(PID), I'm thinking about
> defining a function that logs the plan of a running query and
> untruncated query string on the specified PID in the same way
> as below.
>
> postgres=# SELECT * FROM pg_log_current_plan(2155192);
> pg_log_current_plan
> ---------------------
> t
> (1 row)
+1 for the idea. I did not read the complete patch but while reading
through the patch, I noticed that you using elevel as LOG for printing
the stack trace. But I think the backend whose pid you have passed,
the connected client to that backend might not have superuser
privileges and if you use elevel LOG then that message will be sent to
that connected client as well and I don't think that is secure. So
can we use LOG_SERVER_ONLY so that we can prevent
it from sending to the client.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
@ 2021-05-13 09:27 ` Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-05-13 09:27 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 2:44 PM Dilip Kumar <[email protected]> wrote:
> +1 for the idea. I did not read the complete patch but while reading
> through the patch, I noticed that you using elevel as LOG for printing
> the stack trace. But I think the backend whose pid you have passed,
> the connected client to that backend might not have superuser
> privileges and if you use elevel LOG then that message will be sent to
> that connected client as well and I don't think that is secure. So
> can we use LOG_SERVER_ONLY so that we can prevent
> it from sending to the client.
True, we should use LOG_SERVER_ONLY and not send any logs to the client.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-05-13 09:36 ` Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: Bharath Rupireddy @ 2021-05-13 09:36 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy
<[email protected]> wrote:
> On Thu, May 13, 2021 at 2:44 PM Dilip Kumar <[email protected]> wrote:
> > +1 for the idea. I did not read the complete patch but while reading
> > through the patch, I noticed that you using elevel as LOG for printing
> > the stack trace. But I think the backend whose pid you have passed,
> > the connected client to that backend might not have superuser
> > privileges and if you use elevel LOG then that message will be sent to
> > that connected client as well and I don't think that is secure. So
> > can we use LOG_SERVER_ONLY so that we can prevent
> > it from sending to the client.
>
> True, we should use LOG_SERVER_ONLY and not send any logs to the client.
I further tend to think that, is it correct to log queries with LOG
level when log_statement GUC is set? Or should it also be
LOG_SERVER_ONLY?
/* Log immediately if dictated by log_statement */
if (check_log_statement(parsetree_list))
{
ereport(LOG,
(errmsg("statement: %s", query_string),
errhidestmt(true),
errdetail_execute(parsetree_list)));
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-05-13 09:49 ` Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Dilip Kumar @ 2021-05-13 09:49 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 3:06 PM Bharath Rupireddy
<[email protected]> wrote:
>
> On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy
> <[email protected]> wrote:
> > On Thu, May 13, 2021 at 2:44 PM Dilip Kumar <[email protected]> wrote:
> > > +1 for the idea. I did not read the complete patch but while reading
> > > through the patch, I noticed that you using elevel as LOG for printing
> > > the stack trace. But I think the backend whose pid you have passed,
> > > the connected client to that backend might not have superuser
> > > privileges and if you use elevel LOG then that message will be sent to
> > > that connected client as well and I don't think that is secure. So
> > > can we use LOG_SERVER_ONLY so that we can prevent
> > > it from sending to the client.
> >
> > True, we should use LOG_SERVER_ONLY and not send any logs to the client.
>
> I further tend to think that, is it correct to log queries with LOG
> level when log_statement GUC is set? Or should it also be
> LOG_SERVER_ONLY?
>
> /* Log immediately if dictated by log_statement */
> if (check_log_statement(parsetree_list))
> {
> ereport(LOG,
> (errmsg("statement: %s", query_string),
> errhidestmt(true),
> errdetail_execute(parsetree_list)));
>
What is your argument behind logging it with LOG? I mean we are
sending the signal to all the backend and some backend might have the
client who is not connected as a superuser so sending the plan to
those clients is not a good idea from a security perspective.
Anyways, LOG_SERVER_ONLY is not an exposed logging level it is used
for an internal purpose. So IMHO it should be logged with
LOG_SERVER_ONLY level.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
@ 2021-05-13 10:46 ` Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-05-13 10:46 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 3:20 PM Dilip Kumar <[email protected]> wrote:
>
> On Thu, May 13, 2021 at 3:06 PM Bharath Rupireddy
> <[email protected]> wrote:
> >
> > On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy
> > <[email protected]> wrote:
> > > On Thu, May 13, 2021 at 2:44 PM Dilip Kumar <[email protected]> wrote:
> > > > +1 for the idea. I did not read the complete patch but while reading
> > > > through the patch, I noticed that you using elevel as LOG for printing
> > > > the stack trace. But I think the backend whose pid you have passed,
> > > > the connected client to that backend might not have superuser
> > > > privileges and if you use elevel LOG then that message will be sent to
> > > > that connected client as well and I don't think that is secure. So
> > > > can we use LOG_SERVER_ONLY so that we can prevent
> > > > it from sending to the client.
> > >
> > > True, we should use LOG_SERVER_ONLY and not send any logs to the client.
> >
> > I further tend to think that, is it correct to log queries with LOG
> > level when log_statement GUC is set? Or should it also be
> > LOG_SERVER_ONLY?
> >
> > /* Log immediately if dictated by log_statement */
> > if (check_log_statement(parsetree_list))
> > {
> > ereport(LOG,
> > (errmsg("statement: %s", query_string),
> > errhidestmt(true),
> > errdetail_execute(parsetree_list)));
>
> What is your argument behind logging it with LOG? I mean we are
> sending the signal to all the backend and some backend might have the
> client who is not connected as a superuser so sending the plan to
> those clients is not a good idea from a security perspective.
> Anyways, LOG_SERVER_ONLY is not an exposed logging level it is used
> for an internal purpose. So IMHO it should be logged with
> LOG_SERVER_ONLY level.
I'm saying that - currently, queries are logged with LOG level when
the log_statement GUC is set. The queries might be sent to the
non-superuser clients. So, your point of "sending the plan to those
clients is not a good idea from a security perspective" gets violated
right? Should the log level be changed(in the below code) from "LOG"
to "LOG_SERVER_ONLY"? I think we can discuss this separately so as not
to sidetrack the main feature.
/* Log immediately if dictated by log_statement */
if (check_log_statement(parsetree_list))
{
ereport(LOG,
(errmsg("statement: %s", query_string),
errhidestmt(true),
errdetail_execute(parsetree_list)));
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-05-13 11:43 ` Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Dilip Kumar @ 2021-05-13 11:43 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy
<[email protected]> wrote:
>
> I'm saying that - currently, queries are logged with LOG level when
> the log_statement GUC is set. The queries might be sent to the
> non-superuser clients. So, your point of "sending the plan to those
> clients is not a good idea from a security perspective" gets violated
> right? Should the log level be changed(in the below code) from "LOG"
> to "LOG_SERVER_ONLY"? I think we can discuss this separately so as not
> to sidetrack the main feature.
>
> /* Log immediately if dictated by log_statement */
> if (check_log_statement(parsetree_list))
> {
> ereport(LOG,
> (errmsg("statement: %s", query_string),
> errhidestmt(true),
> errdetail_execute(parsetree_list)));
>
Yes, that was my exact point, that in this particular code log with
LOG_SERVER_ONLY.
Like this.
/* Log immediately if dictated by log_statement */
if (check_log_statement(parsetree_list))
{
ereport(LOG_SERVER_ONLY,
.....
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
@ 2021-05-13 11:45 ` Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-05-13 11:45 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 5:14 PM Dilip Kumar <[email protected]> wrote:
>
> On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy
> <[email protected]> wrote:
> >
> > I'm saying that - currently, queries are logged with LOG level when
> > the log_statement GUC is set. The queries might be sent to the
> > non-superuser clients. So, your point of "sending the plan to those
> > clients is not a good idea from a security perspective" gets violated
> > right? Should the log level be changed(in the below code) from "LOG"
> > to "LOG_SERVER_ONLY"? I think we can discuss this separately so as not
> > to sidetrack the main feature.
> >
> > /* Log immediately if dictated by log_statement */
> > if (check_log_statement(parsetree_list))
> > {
> > ereport(LOG,
> > (errmsg("statement: %s", query_string),
> > errhidestmt(true),
> > errdetail_execute(parsetree_list)));
> >
>
> Yes, that was my exact point, that in this particular code log with
> LOG_SERVER_ONLY.
>
> Like this.
> /* Log immediately if dictated by log_statement */
> if (check_log_statement(parsetree_list))
> {
> ereport(LOG_SERVER_ONLY,
Agree, but let's discuss that in a separate thread.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-05-13 11:48 ` Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Dilip Kumar @ 2021-05-13 11:48 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy
<[email protected]> wrote:
>
> On Thu, May 13, 2021 at 5:14 PM Dilip Kumar <[email protected]> wrote:
> >
> > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy
> > <[email protected]> wrote:
> > >
> > > I'm saying that - currently, queries are logged with LOG level when
> > > the log_statement GUC is set. The queries might be sent to the
> > > non-superuser clients. So, your point of "sending the plan to those
> > > clients is not a good idea from a security perspective" gets violated
> > > right? Should the log level be changed(in the below code) from "LOG"
> > > to "LOG_SERVER_ONLY"? I think we can discuss this separately so as not
> > > to sidetrack the main feature.
> > >
> > > /* Log immediately if dictated by log_statement */
> > > if (check_log_statement(parsetree_list))
> > > {
> > > ereport(LOG,
> > > (errmsg("statement: %s", query_string),
> > > errhidestmt(true),
> > > errdetail_execute(parsetree_list)));
> > >
> >
> > Yes, that was my exact point, that in this particular code log with
> > LOG_SERVER_ONLY.
> >
> > Like this.
> > /* Log immediately if dictated by log_statement */
> > if (check_log_statement(parsetree_list))
> > {
> > ereport(LOG_SERVER_ONLY,
>
> Agree, but let's discuss that in a separate thread.
Did not understand why separate thread? this is part of this thread
no? but anyways now everyone agreed that we will log with
LOG_SERVER_ONLY.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
@ 2021-05-13 12:57 ` Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Dilip Kumar @ 2021-05-13 12:57 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: torikoshia <[email protected]>; pgsql-hackers
On Thu, May 13, 2021 at 5:18 PM Dilip Kumar <[email protected]> wrote:
>
> On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy
> <[email protected]> wrote:
> >
> > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar <[email protected]> wrote:
> > >
> > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy
> > > <[email protected]> wrote:
> > > >
> > > > I'm saying that - currently, queries are logged with LOG level when
> > > > the log_statement GUC is set. The queries might be sent to the
> > > > non-superuser clients. So, your point of "sending the plan to those
> > > > clients is not a good idea from a security perspective" gets violated
> > > > right? Should the log level be changed(in the below code) from "LOG"
> > > > to "LOG_SERVER_ONLY"? I think we can discuss this separately so as not
> > > > to sidetrack the main feature.
> > > >
> > > > /* Log immediately if dictated by log_statement */
> > > > if (check_log_statement(parsetree_list))
> > > > {
> > > > ereport(LOG,
> > > > (errmsg("statement: %s", query_string),
> > > > errhidestmt(true),
> > > > errdetail_execute(parsetree_list)));
> > > >
> > >
> > > Yes, that was my exact point, that in this particular code log with
> > > LOG_SERVER_ONLY.
> > >
> > > Like this.
> > > /* Log immediately if dictated by log_statement */
> > > if (check_log_statement(parsetree_list))
> > > {
> > > ereport(LOG_SERVER_ONLY,
> >
> > Agree, but let's discuss that in a separate thread.
>
> Did not understand why separate thread? this is part of this thread
> no? but anyways now everyone agreed that we will log with
> LOG_SERVER_ONLY.
Bharat offlist pointed to me that here he was talking about another
log that is logging the query and not specific to this patch, so let's
not discuss this point here.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
@ 2021-05-28 06:51 ` torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-05-28 06:51 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; +Cc: Bharath Rupireddy <[email protected]>; pgsql-hackers
On 2021-05-13 21:57, Dilip Kumar wrote:
> On Thu, May 13, 2021 at 5:18 PM Dilip Kumar <[email protected]>
> wrote:
>>
>> On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy
>> <[email protected]> wrote:
>> >
>> > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar <[email protected]> wrote:
>> > >
>> > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy
>> > > <[email protected]> wrote:
>> > > >
>> > > > I'm saying that - currently, queries are logged with LOG level when
>> > > > the log_statement GUC is set. The queries might be sent to the
>> > > > non-superuser clients. So, your point of "sending the plan to those
>> > > > clients is not a good idea from a security perspective" gets violated
>> > > > right? Should the log level be changed(in the below code) from "LOG"
>> > > > to "LOG_SERVER_ONLY"? I think we can discuss this separately so as not
>> > > > to sidetrack the main feature.
>> > > >
>> > > > /* Log immediately if dictated by log_statement */
>> > > > if (check_log_statement(parsetree_list))
>> > > > {
>> > > > ereport(LOG,
>> > > > (errmsg("statement: %s", query_string),
>> > > > errhidestmt(true),
>> > > > errdetail_execute(parsetree_list)));
>> > > >
>> > >
>> > > Yes, that was my exact point, that in this particular code log with
>> > > LOG_SERVER_ONLY.
>> > >
>> > > Like this.
>> > > /* Log immediately if dictated by log_statement */
>> > > if (check_log_statement(parsetree_list))
>> > > {
>> > > ereport(LOG_SERVER_ONLY,
>> >
>> > Agree, but let's discuss that in a separate thread.
>>
>> Did not understand why separate thread? this is part of this thread
>> no? but anyways now everyone agreed that we will log with
>> LOG_SERVER_ONLY.
Modified elevel from LOG to LOG_SERVER_ONLY.
I also modified the patch to log JIT Summary and GUC settings
information.
If there is other useful information to log, I would appreciate it if
you could point it out.
> Bharat offlist pointed to me that here he was talking about another
> log that is logging the query and not specific to this patch, so let's
> not discuss this point here.
Thanks for sharing the situation!
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v2-0001-log-running-query-plan.patch (11.5K, ../../[email protected]/2-v2-0001-log-running-query-plan.patch)
download | inline diff:
From 02297b02bb305d2b8b67f86979fb9252ef5e1264 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Fri, 28 May 2021 13:06:06 +0900
Subject: [PATCH v2] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_plan() function that requests to log the plan
of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
---
doc/src/sgml/func.sgml | 21 +++++
src/backend/commands/explain.c | 114 +++++++++++++++++++++++++++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 ++
src/include/commands/explain.h | 2 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
9 files changed, 154 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 08b07f561e..399fa12aa2 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24940,6 +24940,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_plan</primary>
+ </indexterm>
+ <function>pg_log_current_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the untruncated query string and its plan for
+ the query currently running on the backend with the specified
+ process ID.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 9a60865d19..9d934f28a8 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -4928,3 +4931,114 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the
+ * plan of a currently running query.
+ *
+ * All the actual work is deferred to ProcessLogExplainInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es = NewExplainState();
+
+ LogCurrentPlanPending = false;
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+
+ if (ActivePortal && ActivePortal->queryDesc != NULL)
+ {
+ ExplainQueryText(es, ActivePortal->queryDesc);
+ ExplainPrintPlan(es, ActivePortal->queryDesc);
+ ExplainPrintJITSummary(es, ActivePortal->queryDesc);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("logging plan of the running query on PID %d\n%s",
+ MyProcPid,
+ es->str->data),
+ errhidestmt(true)));
+ }
+ else
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("PID %d is not executing queries now",
+ MyProcPid)));
+}
+
+/*
+ * pg_log_current_plan
+ * Signal a backend process to log plan the of running query.
+ *
+ * Only superusers are allowed to signal to log plan because any users to
+ * issue this request at an unbounded rate would cause lots of log messages
+ * and which can lead to denial of service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal handler,
+ * which causes the next CHECK_FOR_INTERRUPTS() to log plan.
+ */
+Datum
+pg_log_current_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to look into plans of long running queries, that it might end its
+ * own first and its plan is not logged is not a problem.
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ /* Only allow superusers to log plan of the running queries. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log plan of the running query")));
+
+ if (SendProcSignal(pid, PROCSIG_LOG_CURRENT_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..9b9653544a 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..ac8ea67e81 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index acbcae4607..a6be1ab0b5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7975,6 +7975,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..fa81beb553 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -124,4 +124,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 4dc343cbc5..355e3db622 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
base-commit: 9afdea982420f9672b88e5c17d1ee8eec64105fc
--
2.18.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-06-09 07:44 ` torikoshia <[email protected]>
2021-06-09 14:04 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: torikoshia @ 2021-06-09 07:44 UTC (permalink / raw)
To: Dilip Kumar <[email protected]>; pgsql-hackers; +Cc: Bharath Rupireddy <[email protected]>
On 2021-05-28 15:51, torikoshia wrote:
> On 2021-05-13 21:57, Dilip Kumar wrote:
>> On Thu, May 13, 2021 at 5:18 PM Dilip Kumar <[email protected]>
>> wrote:
>>>
>>> On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy
>>> <[email protected]> wrote:
>>> >
>>> > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar <[email protected]> wrote:
>>> > >
>>> > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy
>>> > > <[email protected]> wrote:
>>> > > >
>>> > > > I'm saying that - currently, queries are logged with LOG level when
>>> > > > the log_statement GUC is set. The queries might be sent to the
>>> > > > non-superuser clients. So, your point of "sending the plan to those
>>> > > > clients is not a good idea from a security perspective" gets violated
>>> > > > right? Should the log level be changed(in the below code) from "LOG"
>>> > > > to "LOG_SERVER_ONLY"? I think we can discuss this separately so as not
>>> > > > to sidetrack the main feature.
>>> > > >
>>> > > > /* Log immediately if dictated by log_statement */
>>> > > > if (check_log_statement(parsetree_list))
>>> > > > {
>>> > > > ereport(LOG,
>>> > > > (errmsg("statement: %s", query_string),
>>> > > > errhidestmt(true),
>>> > > > errdetail_execute(parsetree_list)));
>>> > > >
>>> > >
>>> > > Yes, that was my exact point, that in this particular code log with
>>> > > LOG_SERVER_ONLY.
>>> > >
>>> > > Like this.
>>> > > /* Log immediately if dictated by log_statement */
>>> > > if (check_log_statement(parsetree_list))
>>> > > {
>>> > > ereport(LOG_SERVER_ONLY,
>>> >
>>> > Agree, but let's discuss that in a separate thread.
>>>
>>> Did not understand why separate thread? this is part of this thread
>>> no? but anyways now everyone agreed that we will log with
>>> LOG_SERVER_ONLY.
>
> Modified elevel from LOG to LOG_SERVER_ONLY.
>
> I also modified the patch to log JIT Summary and GUC settings
> information.
> If there is other useful information to log, I would appreciate it if
> you could point it out.
Updated the patch.
- reordered superuser check which was pointed out in another thread[1]
- added a regression test
[1] https://postgr.es/m/[email protected]
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v3-0001-log-running-query-plan.patch (13.5K, ../../[email protected]/2-v3-0001-log-running-query-plan.patch)
download | inline diff:
From 7ad9a280b6f74e4718293863716046c02b0a3835 Mon Sep 17 00:00:00 2001
From: atorik <[email protected]>
Date: Wed, 9 Jun 2021 15:03:39 +0900
Subject: [PATCH v3] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_plan() function that requests to log the plan
of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
---
doc/src/sgml/func.sgml | 21 ++++
src/backend/commands/explain.c | 116 +++++++++++++++++++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 2 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/test/regress/expected/misc_functions.out | 10 +-
src/test/regress/sql/misc_functions.sql | 6 +-
11 files changed, 167 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 08b07f561e..399fa12aa2 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24940,6 +24940,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_plan</primary>
+ </indexterm>
+ <function>pg_log_current_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the untruncated query string and its plan for
+ the query currently running on the backend with the specified
+ process ID.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 9a60865d19..1cea557764 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -4928,3 +4931,116 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the
+ * plan of a currently running query.
+ *
+ * All the actual work is deferred to ProcessLogExplainInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es = NewExplainState();
+
+ LogCurrentPlanPending = false;
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+
+ if (ActivePortal && ActivePortal->queryDesc != NULL)
+ {
+ ExplainQueryText(es, ActivePortal->queryDesc);
+ ExplainPrintPlan(es, ActivePortal->queryDesc);
+ ExplainPrintJITSummary(es, ActivePortal->queryDesc);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("logging plan of the running query on PID %d\n%s",
+ MyProcPid,
+ es->str->data),
+ errhidestmt(true)));
+ }
+ else
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("PID %d is not executing queries now",
+ MyProcPid)));
+}
+
+/*
+ * pg_log_current_plan
+ * Signal a backend process to log plan the of running query.
+ *
+ * Only superusers are allowed to signal to log plan because any users to
+ * issue this request at an unbounded rate would cause lots of log messages
+ * and which can lead to denial of service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal handler,
+ * which causes the next CHECK_FOR_INTERRUPTS() to log plan.
+ */
+Datum
+pg_log_current_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ /* Only allow superusers to log plan of the running queries. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log plan of the running query")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to look into plans of long running queries, that it might end its
+ * own first and its plan is not logged is not a problem.
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_CURRENT_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..9b9653544a 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..ac8ea67e81 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index acbcae4607..a6be1ab0b5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7975,6 +7975,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..fa81beb553 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -124,4 +124,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 4dc343cbc5..355e3db622 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..4c6159a11c 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,9 +134,9 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- Some functions output what you want to the log.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail.
--
@@ -146,6 +146,12 @@ SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
t
(1 row)
+SELECT * FROM pg_log_current_plan(pg_backend_pid());
+ pg_log_current_plan
+---------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..7aed52f437 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,15 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- Some functions output what you want to the log.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail.
--
SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT * FROM pg_log_current_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: be90098907475f3cfff7dc6d590641b9c2dae081
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-06-09 14:04 ` Fujii Masao <[email protected]>
2021-06-10 02:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Fujii Masao @ 2021-06-09 14:04 UTC (permalink / raw)
To: torikoshia <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers; +Cc: Bharath Rupireddy <[email protected]>
On 2021/06/09 16:44, torikoshia wrote:
> Updated the patch.
Thanks for updating the patch!
auto_explain can log the plan of even nested statement
if auto_explain.log_nested_statements is enabled.
But ISTM that pg_log_current_plan() cannot log that plan.
Is this intentional?
I think that it's better to make pg_log_current_plan() log
the plan of even nested statement.
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
Since pg_log_current_plan() is usually used to investigate and
trouble-shoot the long running queries, IMO it's better to
enable es->verbose by default and get additional information
about the queries. Thought?
+ * pg_log_current_plan
+ * Signal a backend process to log plan the of running query.
"plan the of" is typo?
+ * Only superusers are allowed to signal to log plan because any users to
+ * issue this request at an unbounded rate would cause lots of log messages
+ * and which can lead to denial of service.
"because any users" should be "because allowing any users"
like the comment for pg_log_backend_memory_contexts()?
+ * All the actual work is deferred to ProcessLogExplainInterrupt(),
"ProcessLogExplainInterrupt()" should be "ProcessLogCurrentPlanInterrupt()"?
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 14:04 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
@ 2021-06-10 02:09 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2021-06-10 02:09 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; pgsql-hackers
On 2021-06-09 23:04, Fujii Masao wrote:
Thanks for your review!
> auto_explain can log the plan of even nested statement
> if auto_explain.log_nested_statements is enabled.
> But ISTM that pg_log_current_plan() cannot log that plan.
> Is this intentional?
> I think that it's better to make pg_log_current_plan() log
> the plan of even nested statement.
+1. It would be better.
But currently plan information is got from ActivePortal and ISTM there
are no easy way to retrieve plan information of nested statements from
ActivePortal.
Anyway I'll do some more research.
I think you are right about the following comments.
I'll fix them.
> + es->format = EXPLAIN_FORMAT_TEXT;
> + es->settings = true;
>
> Since pg_log_current_plan() is usually used to investigate and
> trouble-shoot the long running queries, IMO it's better to
> enable es->verbose by default and get additional information
> about the queries. Thought?
> + * pg_log_current_plan
> + * Signal a backend process to log plan the of running query.
>
> "plan the of" is typo?
>
>
> + * Only superusers are allowed to signal to log plan because any users
> to
> + * issue this request at an unbounded rate would cause lots of log
> messages
> + * and which can lead to denial of service.
>
> "because any users" should be "because allowing any users"
> like the comment for pg_log_backend_memory_contexts()?
>
>
> + * All the actual work is deferred to ProcessLogExplainInterrupt(),
>
> "ProcessLogExplainInterrupt()" should be
> "ProcessLogCurrentPlanInterrupt()"?
>
> Regards,
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-06-10 16:20 ` Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-06-10 16:20 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Dilip Kumar <[email protected]>; pgsql-hackers
On Wed, Jun 9, 2021 at 1:14 PM torikoshia <[email protected]> wrote:
> Updated the patch.
Thanks for the patch. Here are some comments on v3 patch:
1) We could just say "Requests to log query plan of the presently
running query of a given backend along with an untruncated query
string in the server logs."
Instead of
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
2) It's better to do below, for reference see how pg_backend_pid,
pg_terminate_backend, pg_relpages and so on are used in the tests.
+SELECT pg_log_current_plan(pg_backend_pid());
rather than using the function in the FROM clause.
+SELECT * FROM pg_log_current_plan(pg_backend_pid());
If okay, also change it for pg_log_backend_memory_contexts.
3) Since most of the code looks same in pg_log_backend_memory_contexts
and pg_log_current_plan, I think we can have a common function
something like below:
bool
SendProcSignalForLogInfo(ProcSignalReason reason)
{
Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT || reason ==
PROCSIG_LOG_CURRENT_PLAN);
if (!superuser())
{
if (reason == PROCSIG_LOG_MEMORY_CONTEXT)
errmsg("must be a superuser to log memory contexts")
else if (reason == PROCSIG_LOG_CURRENT_PLAN)
errmsg("must be a superuser to log plan of the running query")
}
if (SendProcSignal(pid, reason, proc->backendId) < 0)
{
}
}
Then we could just do:
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
PG_RETURN_BOOL(SendProcSignalForLogInfo(PROCSIG_LOG_MEMORY_CONTEXT));
}
Datum
pg_log_current_plan(PG_FUNCTION_ARGS)
{
PG_RETURN_BOOL(SendProcSignalForLogInfo(PROCSIG_LOG_CURRENT_PLAN));
}
We can have SendProcSignalForLogInfo function defined in procsignal.c
and procsignal.h
4) I think we can have a sample function usage and how it returns true
value, how the plan looks for a simple query(select 1 or some other
simple/complex generic query or simply select
pg_log_current_plan(pg_backend_pid());) in the documentation, much
like pg_log_backend_memory_contexts.
5) Instead of just showing the true return value of the function
pg_log_current_plan in the sql test, which just shows that the signal
is sent, but it doesn't mean that the backend has processed that
signal and logged the plan. I think we can add the test using TAP
framework, one
6) Do we unnecessarily need to signal the processes such as autovacuum
launcher/workers, logical replication launcher/workers, wal senders,
wal receivers and so on. only to emit a "PID %d is not executing
queries now" message? Moreover, those processes will be waiting in
loops for timeouts to occur, then as soon as they wake up do they need
to process this extra uninformative signal?
We could choose to not signal those processes at all which might or
might not be possible.
Otherwise, we could just emit messages like "XXXX process cannot run a
query" in ProcessInterrupts.
7)Instead of
(errmsg("logging plan of the running query on PID %d\n%s",
how about below?
(errmsg("plan of the query running on backend with PID %d is:\n%s",
8) Instead of
errmsg("PID %d is not executing queries now")
how about below?
errmsg("Backend with PID %d is not running a query")
9) We could just do:
void
ProcessLogCurrentPlanInterrupt(void)
{
ExplainState *es;
LogCurrentPlanPending = false;
if (!ActivePortal || !ActivePortal->queryDesc)
errmsg("PID %d is not executing queries now");
es = NewExplainState();
ExplainQueryText();
ExplainPrintPlan();
10) How about renaming the function pg_log_current_plan to
pg_log_query_plan or pg_log_current_query_plan?
11) What happens if pg_log_current_plan is called for a parallel worker?
With Regards,
Bharath Rupireddy.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-06-14 12:18 ` torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-06-14 12:18 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-06-11 01:20, Bharath Rupireddy wrote:
Thanks for your review!
> On Wed, Jun 9, 2021 at 1:14 PM torikoshia <[email protected]>
> wrote:
>> Updated the patch.
>
> Thanks for the patch. Here are some comments on v3 patch:
>
> 1) We could just say "Requests to log query plan of the presently
> running query of a given backend along with an untruncated query
> string in the server logs."
> Instead of
> + They will be logged at <literal>LOG</literal> message level
> and
> + will appear in the server log based on the log
> + configuration set (See <xref
> linkend="runtime-config-logging"/>
Actually this explanation is almost the same as that of
pg_log_backend_memory_contexts().
Do you think we should change both of them?
I think it may be too detailed but accurate.
> 2) It's better to do below, for reference see how pg_backend_pid,
> pg_terminate_backend, pg_relpages and so on are used in the tests.
> +SELECT pg_log_current_plan(pg_backend_pid());
> rather than using the function in the FROM clause.
> +SELECT * FROM pg_log_current_plan(pg_backend_pid());
> If okay, also change it for pg_log_backend_memory_contexts.
Agreed.
> 3) Since most of the code looks same in pg_log_backend_memory_contexts
> and pg_log_current_plan, I think we can have a common function
> something like below:
Agreed. I'll do some refactoring.
> bool
> SendProcSignalForLogInfo(ProcSignalReason reason)
> {
> Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT || reason ==
> PROCSIG_LOG_CURRENT_PLAN);
>
> if (!superuser())
> {
> if (reason == PROCSIG_LOG_MEMORY_CONTEXT)
> errmsg("must be a superuser to log memory contexts")
> else if (reason == PROCSIG_LOG_CURRENT_PLAN)
> errmsg("must be a superuser to log plan of the running query")
> }
>
> if (SendProcSignal(pid, reason, proc->backendId) < 0)
> {
> }
> }
> Then we could just do:
> Datum
> pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
> {
> PG_RETURN_BOOL(SendProcSignalForLogInfo(PROCSIG_LOG_MEMORY_CONTEXT));
> }
> Datum
> pg_log_current_plan(PG_FUNCTION_ARGS)
> {
> PG_RETURN_BOOL(SendProcSignalForLogInfo(PROCSIG_LOG_CURRENT_PLAN));
> }
> We can have SendProcSignalForLogInfo function defined in procsignal.c
> and procsignal.h
>
> 4) I think we can have a sample function usage and how it returns true
> value, how the plan looks for a simple query(select 1 or some other
> simple/complex generic query or simply select
> pg_log_current_plan(pg_backend_pid());) in the documentation, much
> like pg_log_backend_memory_contexts.
+1.
> 5) Instead of just showing the true return value of the function
> pg_log_current_plan in the sql test, which just shows that the signal
> is sent, but it doesn't mean that the backend has processed that
> signal and logged the plan. I think we can add the test using TAP
> framework, one
I once made a tap test for pg_log_backend_memory_contexts(), but it
seemed an overkill and we agreed that it was appropriate just ensuring
the function working as below discussion.
https://www.postgresql.org/message-id/bbecd722d4f8e261b350186ac4bf68a7%40oss.nttdata.com
> 6) Do we unnecessarily need to signal the processes such as autovacuum
> launcher/workers, logical replication launcher/workers, wal senders,
> wal receivers and so on. only to emit a "PID %d is not executing
> queries now" message? Moreover, those processes will be waiting in
> loops for timeouts to occur, then as soon as they wake up do they need
> to process this extra uninformative signal?
> We could choose to not signal those processes at all which might or
> might not be possible.
> Otherwise, we could just emit messages like "XXXX process cannot run a
> query" in ProcessInterrupts.
Agreed.
However it needs to distinguish backends which can execute queries and
other processes such as autovacuum launcher, I don't come up with
easy ways to do so.
I'm going to think about it.
> 7)Instead of
> (errmsg("logging plan of the running query on PID %d\n%s",
> how about below?
> (errmsg("plan of the query running on backend with PID %d is:\n%s",
+1.
> 8) Instead of
> errmsg("PID %d is not executing queries now")
> how about below?
> errmsg("Backend with PID %d is not running a query")
+1.
>
> 9) We could just do:
> void
> ProcessLogCurrentPlanInterrupt(void)
> {
> ExplainState *es;
> LogCurrentPlanPending = false;
> if (!ActivePortal || !ActivePortal->queryDesc)
> errmsg("PID %d is not executing queries now");
> es = NewExplainState();
> ExplainQueryText();
> ExplainPrintPlan();
>
> 10) How about renaming the function pg_log_current_plan to
> pg_log_query_plan or pg_log_current_query_plan?
+1.
> 11) What happens if pg_log_current_plan is called for a parallel
> worker?
AFAIU Parallel worker doesn't have ActivePortal, so it would always
emit the message 'PID %d is not executing queries now'.
As 6), it would be better to distinguish the parallel worker and normal
backend.
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-06-15 04:27 ` Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-06-15 04:27 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Dilip Kumar <[email protected]>; pgsql-hackers
On Mon, Jun 14, 2021 at 5:48 PM torikoshia <[email protected]> wrote:
> > 1) We could just say "Requests to log query plan of the presently
> > running query of a given backend along with an untruncated query
> > string in the server logs."
> > Instead of
> > + They will be logged at <literal>LOG</literal> message level
> > and
> > + will appear in the server log based on the log
> > + configuration set (See <xref
> > linkend="runtime-config-logging"/>
>
> Actually this explanation is almost the same as that of
> pg_log_backend_memory_contexts().
> Do you think we should change both of them?
> I think it may be too detailed but accurate.
I withdraw my comment. We can keep the explanation similar to
pg_log_backend_memory_contexts as it was agreed upon and committed
text. If the wordings are similar, then it will be easier for users to
understand the documentation.
> > 5) Instead of just showing the true return value of the function
> > pg_log_current_plan in the sql test, which just shows that the signal
> > is sent, but it doesn't mean that the backend has processed that
> > signal and logged the plan. I think we can add the test using TAP
> > framework, one
>
> I once made a tap test for pg_log_backend_memory_contexts(), but it
> seemed an overkill and we agreed that it was appropriate just ensuring
> the function working as below discussion.
>
> https://www.postgresql.org/message-id/bbecd722d4f8e261b350186ac4bf68a7%40oss.nttdata.com
Okay. I withdraw my comment.
> > 6) Do we unnecessarily need to signal the processes such as autovacuum
> > launcher/workers, logical replication launcher/workers, wal senders,
> > wal receivers and so on. only to emit a "PID %d is not executing
> > queries now" message? Moreover, those processes will be waiting in
> > loops for timeouts to occur, then as soon as they wake up do they need
> > to process this extra uninformative signal?
> > We could choose to not signal those processes at all which might or
> > might not be possible.
> > Otherwise, we could just emit messages like "XXXX process cannot run a
> > query" in ProcessInterrupts.
>
> Agreed.
>
> However it needs to distinguish backends which can execute queries and
> other processes such as autovacuum launcher, I don't come up with
> easy ways to do so.
> I'm going to think about it.
I'm not sure if there is any information in the shared memory
accessible to all the backends/sessions that can say a PID is
autovacuum launcher/worker, logical replication launcher/worker or any
other background or parallel worker. If we were to invent a new
mechanism just for addressing the above comment, I would rather choose
to not do that as it seems like an overkill. We can leave it up to the
user whether or not to unnecessarily signal those processes which are
bound to print "PID XXX is not executing queries now" message.
> > 11) What happens if pg_log_current_plan is called for a parallel
> > worker?
>
> AFAIU Parallel worker doesn't have ActivePortal, so it would always
> emit the message 'PID %d is not executing queries now'.
> As 6), it would be better to distinguish the parallel worker and normal
> backend.
As I said, above, I think it will be a bit tough to do. If done, it
seems like an overkill.
With Regards,
Bharath Rupireddy.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-06-16 11:36 ` torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-06-16 11:36 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-06-15 13:27, Bharath Rupireddy wrote:
> On Mon, Jun 14, 2021 at 5:48 PM torikoshia <[email protected]>
> wrote:
>> > 1) We could just say "Requests to log query plan of the presently
>> > running query of a given backend along with an untruncated query
>> > string in the server logs."
>> > Instead of
>> > + They will be logged at <literal>LOG</literal> message level
>> > and
>> > + will appear in the server log based on the log
>> > + configuration set (See <xref
>> > linkend="runtime-config-logging"/>
>>
>> Actually this explanation is almost the same as that of
>> pg_log_backend_memory_contexts().
>> Do you think we should change both of them?
>> I think it may be too detailed but accurate.
>
> I withdraw my comment. We can keep the explanation similar to
> pg_log_backend_memory_contexts as it was agreed upon and committed
> text. If the wordings are similar, then it will be easier for users to
> understand the documentation.
>
>> > 5) Instead of just showing the true return value of the function
>> > pg_log_current_plan in the sql test, which just shows that the signal
>> > is sent, but it doesn't mean that the backend has processed that
>> > signal and logged the plan. I think we can add the test using TAP
>> > framework, one
>>
>> I once made a tap test for pg_log_backend_memory_contexts(), but it
>> seemed an overkill and we agreed that it was appropriate just ensuring
>> the function working as below discussion.
>>
>> https://www.postgresql.org/message-id/bbecd722d4f8e261b350186ac4bf68a7%40oss.nttdata.com
>
> Okay. I withdraw my comment.
>
>> > 6) Do we unnecessarily need to signal the processes such as autovacuum
>> > launcher/workers, logical replication launcher/workers, wal senders,
>> > wal receivers and so on. only to emit a "PID %d is not executing
>> > queries now" message? Moreover, those processes will be waiting in
>> > loops for timeouts to occur, then as soon as they wake up do they need
>> > to process this extra uninformative signal?
>> > We could choose to not signal those processes at all which might or
>> > might not be possible.
>> > Otherwise, we could just emit messages like "XXXX process cannot run a
>> > query" in ProcessInterrupts.
>>
>> Agreed.
>>
>> However it needs to distinguish backends which can execute queries and
>> other processes such as autovacuum launcher, I don't come up with
>> easy ways to do so.
>> I'm going to think about it.
>
> I'm not sure if there is any information in the shared memory
> accessible to all the backends/sessions that can say a PID is
> autovacuum launcher/worker, logical replication launcher/worker or any
> other background or parallel worker.
As far as I looked around, there seems no easy ways to do so.
> If we were to invent a new
> mechanism just for addressing the above comment, I would rather choose
> to not do that as it seems like an overkill. We can leave it up to the
> user whether or not to unnecessarily signal those processes which are
> bound to print "PID XXX is not executing queries now" message.
+1. I'm going to proceed in this direction.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-06-22 02:30 ` torikoshia <[email protected]>
2021-07-01 06:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: torikoshia @ 2021-06-22 02:30 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; [email protected]; +Cc: Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-06-16 20:36, torikoshia wrote:
>> other background or parallel worker.
>
> As far as I looked around, there seems no easy ways to do so.
>
>> If we were to invent a new
>> mechanism just for addressing the above comment, I would rather choose
>> to not do that as it seems like an overkill. We can leave it up to the
>> user whether or not to unnecessarily signal those processes which are
>> bound to print "PID XXX is not executing queries now" message.
>
> +1. I'm going to proceed in this direction.
Updated the patch.
On Thu, Jun 10, 2021 at 11:09 AM torikoshia <[email protected]>
wrote:
> On 2021-06-09 23:04, Fujii Masao wrote:
> > auto_explain can log the plan of even nested statement
> > if auto_explain.log_nested_statements is enabled.
> > But ISTM that pg_log_current_plan() cannot log that plan.
> > Is this intentional?
>
> > I think that it's better to make pg_log_current_plan() log
> > the plan of even nested statement.
>
> +1. It would be better.
> But currently plan information is got from ActivePortal and ISTM there
> are no easy way to retrieve plan information of nested statements from
> ActivePortal.
> Anyway I'll do some more research.
I haven't found a proper way yet but it seems necessary to use something
other than ActivePortal and I'm now thinking this could be a separate
patch in the future.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v4-0001-log-running-query-plan.patch (18.7K, ../../[email protected]/2-v4-0001-log-running-query-plan.patch)
download | inline diff:
From 7ad9a280b6f74e4718293863716046c02b0a3835 Mon Sep 17 00:00:00 2001
From: atorik <[email protected]>
Date: Tue, 22 Jun 2021 10:03:39 +0900
Subject: [PATCH v4] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes and comments of pg_log_current_query_plan()
are the same with pg_log_backend_memory_contexts(), this
patch also refactors them to make them common.
Reviewd-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar
---
doc/src/sgml/func.sgml | 44 ++++++++++++
src/backend/commands/explain.c | 75 ++++++++++++++++++++
src/backend/storage/ipc/procsignal.c | 66 +++++++++++++++++
src/backend/tcop/postgres.c | 4 ++
src/backend/utils/adt/mcxtfuncs.c | 51 +------------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 ++
src/include/commands/explain.h | 2 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 3 +
src/test/regress/expected/misc_functions.out | 12 +++-
src/test/regress/sql/misc_functions.sql | 8 +--
12 files changed, 217 insertions(+), 56 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6388385edc..dad2d34a04 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24940,6 +24940,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the untruncated query string and its plan for
+ the query currently running on the backend with the specified
+ process ID.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25053,6 +25074,29 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>FORMAT TEXT</literal>
+and <literal>VEBOSE</literal> are used in the <command>EXPLAIN</command> command.
+For example:
+<screen>
+LOG: plan of the query running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..26394.00 rows=1000000 width=97)
+ Output: aid, bid, abalance, filler
+</screen>
+ Note that nested statements (statements executed inside a function) is not
+ considered for logging. Only top-level query plans are logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e81b990092..c568a2e234 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -4922,3 +4925,75 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es = NewExplainState();
+
+ LogCurrentPlanPending = false;
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+
+ if (!ActivePortal || !ActivePortal->queryDesc)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ ExplainQueryText(es, ActivePortal->queryDesc);
+ ExplainPrintPlan(es, ActivePortal->queryDesc);
+ ExplainPrintJITSummary(es, ActivePortal->queryDesc);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid,
+ es->str->data),
+ errhidestmt(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid = PG_GETARG_INT32(0);
+
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN));
+}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..fe9ea90222 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -27,6 +28,7 @@
#include "storage/ipc.h"
#include "storage/latch.h"
#include "storage/proc.h"
+#include "storage/procarray.h"
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
@@ -312,6 +314,67 @@ SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
return -1;
}
+/*
+ * SendProcSignalForLogInfo
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
+
/*
* EmitProcSignalBarrier
* Send a signal to every Postgres process
@@ -661,6 +724,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..ac8ea67e81 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..efb9942c5f 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,7 +18,6 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/builtins.h"
@@ -161,57 +160,11 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
-
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid = PG_GETARG_INT32(0);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT));
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fde251fa4f..629d393828 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7970,6 +7970,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..fa81beb553 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -124,4 +124,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 4dc343cbc5..355e3db622 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..92290c2ed9 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
@@ -66,6 +67,8 @@ extern void ProcSignalShmemInit(void);
extern void ProcSignalInit(int pss_idx);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
+extern bool SendProcSignalForLogInfo(pid_t pid,
+ ProcSignalReason reason);
extern uint64 EmitProcSignalBarrier(ProcSignalBarrierType type);
extern void WaitForProcSignalBarrier(uint64 generation);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..5e9c3cbd8f 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,24 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- Some functions output what you want to the log.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..a3be66ab11 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,15 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- Some functions output what you want to the log.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: bafad2c5b261a1449bed60ebac9e7918ebcc42b4
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-07-01 06:34 ` Fujii Masao <[email protected]>
1 sibling, 0 replies; 124+ messages in thread
From: Fujii Masao @ 2021-07-01 06:34 UTC (permalink / raw)
To: torikoshia <[email protected]>; Bharath Rupireddy <[email protected]>; +Cc: Dilip Kumar <[email protected]>; pgsql-hackers
On 2021/06/22 11:30, torikoshia wrote:
> On Thu, Jun 10, 2021 at 11:09 AM torikoshia <[email protected]> wrote:
>> On 2021-06-09 23:04, Fujii Masao wrote:
>
>> > auto_explain can log the plan of even nested statement
>> > if auto_explain.log_nested_statements is enabled.
>> > But ISTM that pg_log_current_plan() cannot log that plan.
>> > Is this intentional?
>>
>> > I think that it's better to make pg_log_current_plan() log
>> > the plan of even nested statement.
>>
>> +1. It would be better.
>> But currently plan information is got from ActivePortal and ISTM there
>> are no easy way to retrieve plan information of nested statements from
>> ActivePortal.
>> Anyway I'll do some more research.
>
> I haven't found a proper way yet but it seems necessary to use something other than ActivePortal and I'm now thinking this could be a separate patch in the future.
DO $$
BEGIN
PERFORM pg_sleep(100);
END$$;
When I called pg_log_current_query_plan() to send the signal to
the backend executing the above query, I got the following log message.
I think that this is not expected message. I guess this issue happened
because the information about query text and plan is retrieved
from ActivePortal. If this understanding is right, ISTM that we should
implement new mechanism so that we can retrieve those information
even while nested query is being executed.
LOG: backend with PID 42449 is not running a query
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-07-02 14:21 ` Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-07-02 14:21 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Fujii Masao <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On Tue, Jun 22, 2021 at 8:00 AM torikoshia <[email protected]> wrote:
> Updated the patch.
Thanks for the patch. Here are some comments on the v4 patch:
1) Can we do + ExplainState *es = NewExplainState(); and es
assignments after if (!ActivePortal || !ActivePortal->queryDesc), just
to avoid unnecessary call in case of error hit? Also note that, we can
easily hit the error case.
2) It looks like there's an improper indentation. MyProcPid and
es->str->data, should start from the ".
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid,
+ es->str->data),
For reference see errmsg("unrecognized value for EXPLAIN option \"%s\": \"%s\"",
3)I prefer to do this so that any new piece of code can be introduced
in between easily and it will be more readable as well.
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
If okay, please also change for the pg_log_backend_memory_contexts.
4) Extra whitespace before the second line i.e. 2nd line reason should
be aligned with the 1st line reason.
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
5) How about "Requests to log the plan of the query currently running
on the backend with specified process ID along with the untruncated
query string"?
+ Requests to log the untruncated query string and its plan for
+ the query currently running on the backend with the specified
+ process ID.
6) A typo: it is "nested statements (..) are not"
+ Note that nested statements (statements executed inside a function) is not
7) I'm not sure what you mean by "Some functions output what you want
to the log."
--- Memory contexts are logged and they are not returned to the function.
+-- Some functions output what you want to the log.
Instead, can we say "These functions return true if the specified
backend is successfully signaled, otherwise false. Upon receiving the
signal, the backend will log the information to the server log."
Regards,
Bharath Rupireddy.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-07-09 05:05 ` torikoshia <[email protected]>
2021-07-13 14:11 ` Re: RFC: Logging plan of the running query Masahiro Ikeda <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: torikoshia @ 2021-07-09 05:05 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: Fujii Masao <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-07-02 23:21, Bharath Rupireddy wrote:
> On Tue, Jun 22, 2021 at 8:00 AM torikoshia <[email protected]>
> wrote:
>> Updated the patch.
>
> Thanks for the patch. Here are some comments on the v4 patch:
Thanks for your comments and suggestions!
I agree with you and updated the patch.
On Thu, Jul 1, 2021 at 3:34 PM Fujii Masao <[email protected]>
wrote:
> DO $$
> BEGIN
> PERFORM pg_sleep(100);
> END$$;
>
> When I called pg_log_current_query_plan() to send the signal to
> the backend executing the above query, I got the following log message.
> I think that this is not expected message. I guess this issue happened
> because the information about query text and plan is retrieved
> from ActivePortal. If this understanding is right, ISTM that we should
> implement new mechanism so that we can retrieve those information
> even while nested query is being executed.
I'm now working on this comment.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v5-0001-log-running-query-plan.patch (19.2K, ../../[email protected]/2-v5-0001-log-running-query-plan.patch)
download | inline diff:
From 7ad9a280b6f74e4718293863716046c02b0a3835 Mon Sep 17 00:00:00 2001
From: atorik <[email protected]>
Date: Fri, 9 Jul 2021 13:33:02 +0900
Subject: [PATCH v5] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes and comments of pg_log_current_query_plan()
are the same with pg_log_backend_memory_contexts(), this
patch also refactors them to make them common.
Reviewd-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar
---
doc/src/sgml/func.sgml | 44 +++++++++++
src/backend/commands/explain.c | 79 ++++++++++++++++++++
src/backend/storage/ipc/procsignal.c | 66 ++++++++++++++++
src/backend/tcop/postgres.c | 4 +
src/backend/utils/adt/mcxtfuncs.c | 53 ++-----------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 ++
src/include/commands/explain.h | 2 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 3 +
src/test/regress/expected/misc_functions.out | 16 +++-
src/test/regress/sql/misc_functions.sql | 12 +--
12 files changed, 230 insertions(+), 57 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6388385edc..950e32a290 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24940,6 +24940,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25053,6 +25074,29 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>FORMAT TEXT</literal>
+and <literal>VEBOSE</literal> are used in the <command>EXPLAIN</command> command.
+For example:
+<screen>
+LOG: plan of the query running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..26394.00 rows=1000000 width=97)
+ Output: aid, bid, abalance, filler
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only top-level query plans are logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e81b990092..e1cd88e201 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -4922,3 +4925,79 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ LogCurrentPlanPending = false;
+
+ if (!ActivePortal || !ActivePortal->queryDesc)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+
+ ExplainQueryText(es, ActivePortal->queryDesc);
+ ExplainPrintPlan(es, ActivePortal->queryDesc);
+ ExplainPrintJITSummary(es, ActivePortal->queryDesc);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..b89e9a98a5 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -27,6 +28,7 @@
#include "storage/ipc.h"
#include "storage/latch.h"
#include "storage/proc.h"
+#include "storage/procarray.h"
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
@@ -312,6 +314,67 @@ SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
return -1;
}
+/*
+ * SendProcSignalForLogInfo
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
+
/*
* EmitProcSignalBarrier
* Send a signal to every Postgres process
@@ -661,6 +724,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..ac8ea67e81 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..25c621e776 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,7 +18,6 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/builtins.h"
@@ -161,57 +160,15 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid;
+ bool result;
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fde251fa4f..629d393828 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7970,6 +7970,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..fa81beb553 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -124,4 +124,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 4dc343cbc5..355e3db622 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..92290c2ed9 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
@@ -66,6 +67,8 @@ extern void ProcSignalShmemInit(void);
extern void ProcSignalInit(int pss_idx);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
+extern bool SendProcSignalForLogInfo(pid_t pid,
+ ProcSignalReason reason);
extern uint64 EmitProcSignalBarrier(ProcSignalBarrierType type);
extern void WaitForProcSignalBarrier(uint64 generation);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..98017f83a9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,26 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..979a418e8e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,17 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: 387925893edf2a3a30a8ddf2c6474d8a7eb056a5
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-07-13 14:11 ` Masahiro Ikeda <[email protected]>
2021-07-19 02:28 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Masahiro Ikeda @ 2021-07-13 14:11 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Bharath Rupireddy <[email protected]>; Fujii Masao <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On Tue, Jun 22, 2021 at 8:00 AM torikoshia <[email protected]>
wrote:
> Updated the patch.
Hi, torikoshi-san
Thanks for your great work! I'd like to use this feature in v15.
I confirmed that it works with queries I tried and make check-world has
no error.
When I tried this feature, I realized two things. So, I share them.
(1) About output contents
> The format of the query plan is the same as when <literal>FORMAT
> TEXT</literal>
> and <literal>VEBOSE</literal> are used in the
> <command>EXPLAIN</command> command.
> For example:
I think the above needs to add COSTS and SETTINGS options too, and it's
better to use an
example which the SETTINGS option works like the following.
```
2021-07-13 21:59:56 JST 69757 [client backend] LOG: plan of the query
running on backend with PID 69757 is:
Query Text: PREPARE query2 AS SELECT COUNT(*) FROM
pgbench_accounts t1, pgbench_accounts t2;
Aggregate (cost=3750027242.84..3750027242.86 rows=1 width=8)
Output: count(*)
-> Nested Loop (cost=0.84..3125027242.84 rows=250000000000
width=0)
-> Index Only Scan using pgbench_accounts_pkey on
public.pgbench_accounts t1 (cost=0.42..12996.42 rows=500000 width=0)
Output: t1.aid
-> Materialize (cost=0.42..15496.42 rows=500000
width=0)
-> Index Only Scan using pgbench_accounts_pkey on
public.pgbench_accounts t2 (cost=0.42..12996.42 rows=500000 width=0)
Settings: effective_cache_size = '8GB', work_mem = '16MB'
```
(2) About EXPLAIN "BUFFER" option
When I checked EXPLAIN option, I found there is another option "BUFFER"
which can be
used without the "ANALYZE" option.
I'm not sure it's useful because your target use-case is analyzing a
long-running query,
not its planning phase. If so, the planning buffer usage is not so much
useful. But, since
the overhead to output buffer usages is not high and it's used for
debugging use cases,
I wonder it's not a bad idea to output buffer usages too. Thought?
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-13 14:11 ` Re: RFC: Logging plan of the running query Masahiro Ikeda <[email protected]>
@ 2021-07-19 02:28 ` torikoshia <[email protected]>
2021-07-19 06:07 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-07-19 02:28 UTC (permalink / raw)
To: Masahiro Ikeda <[email protected]>; +Cc: Bharath Rupireddy <[email protected]>; Fujii Masao <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On Tue, Jul 13, 2021 at 11:11 PM Masahiro Ikeda
<[email protected]> wrote:
> When I tried this feature, I realized two things. So, I share them.
Thanks for your review!
> (1) About output contents
>
> > The format of the query plan is the same as when <literal>FORMAT
> > TEXT</literal>
> > and <literal>VEBOSE</literal> are used in the
> > <command>EXPLAIN</command> command.
> > For example:
> I think the above needs to add COSTS and SETTINGS options too, and it's
> better to use an
> example which the SETTINGS option works like the following.
Agreed. Updated the patch.
> (2) About EXPLAIN "BUFFER" option
>
> When I checked EXPLAIN option, I found there is another option "BUFFER"
> which can be
> used without the "ANALYZE" option.
>
> I'm not sure it's useful because your target use-case is analyzing a
> long-running query,
> not its planning phase. If so, the planning buffer usage is not so much
> useful. But, since
> the overhead to output buffer usages is not high and it's used for
> debugging use cases,
> I wonder it's not a bad idea to output buffer usages too. Thought?
As you pointed out, I also think it would be useful when queries are
taking a long time in the planning phase.
However, as far as I read ExplainOneQuery(), the buffer usages in the
planner phase are not retrieved by default. They are retrieved only when
BUFFERS is specified in the EXPLAIN.
If we change it to always get the buffer usages and expose them as a
global variable, we can get them through pg_log_current_plan(), but I
think it doesn't pay.
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v6-0001-log-running-query-plan.patch (19.3K, ../../[email protected]/2-v6-0001-log-running-query-plan.patch)
download | inline diff:
From 7ad9a280b6f74e4718293863716046c02b0a3835 Mon Sep 17 00:00:00 2001
From: atorik <[email protected]>
Date: Mon, 19 Jul 2021 10:33:02 +0900
Subject: [PATCH v6] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes and comments of pg_log_current_query_plan()
are the same with pg_log_backend_memory_contexts(), this
patch also refactors them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda
---
doc/src/sgml/func.sgml | 46 ++++++++++++
src/backend/commands/explain.c | 79 ++++++++++++++++++++
src/backend/storage/ipc/procsignal.c | 66 ++++++++++++++++
src/backend/tcop/postgres.c | 4 +
src/backend/utils/adt/mcxtfuncs.c | 53 ++-----------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 ++
src/include/commands/explain.h | 2 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 3 +
src/test/regress/expected/misc_functions.out | 16 +++-
src/test/regress/sql/misc_functions.sql | 12 +--
12 files changed, 232 insertions(+), 57 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ac6347a971..21cd8c9fbb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24940,6 +24940,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25053,6 +25074,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(17793);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only top-level query plans are logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 340db2bac4..cb3ed273f4 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -4921,3 +4924,79 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ LogCurrentPlanPending = false;
+
+ if (!ActivePortal || !ActivePortal->queryDesc)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+
+ ExplainQueryText(es, ActivePortal->queryDesc);
+ ExplainPrintPlan(es, ActivePortal->queryDesc);
+ ExplainPrintJITSummary(es, ActivePortal->queryDesc);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..b89e9a98a5 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -27,6 +28,7 @@
#include "storage/ipc.h"
#include "storage/latch.h"
#include "storage/proc.h"
+#include "storage/procarray.h"
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
@@ -312,6 +314,67 @@ SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
return -1;
}
+/*
+ * SendProcSignalForLogInfo
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
+
/*
* EmitProcSignalBarrier
* Send a signal to every Postgres process
@@ -661,6 +724,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8cea10c901..ac8ea67e81 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..25c621e776 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,7 +18,6 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/builtins.h"
@@ -161,57 +160,15 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid;
+ bool result;
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8bf9d704b7..ed22a17bd2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7974,6 +7974,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..fa81beb553 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -124,4 +124,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 4dc343cbc5..355e3db622 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..92290c2ed9 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
@@ -66,6 +67,8 @@ extern void ProcSignalShmemInit(void);
extern void ProcSignalInit(int pss_idx);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
+extern bool SendProcSignalForLogInfo(pid_t pid,
+ ProcSignalReason reason);
extern uint64 EmitProcSignalBarrier(ProcSignalBarrierType type);
extern void WaitForProcSignalBarrier(uint64 generation);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..98017f83a9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,26 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..979a418e8e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,17 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: f0e21f2f61675f4e56ae53d32ea54d587a7c2257
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-13 14:11 ` Re: RFC: Logging plan of the running query Masahiro Ikeda <[email protected]>
2021-07-19 02:28 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-07-19 06:07 ` Fujii Masao <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: Fujii Masao @ 2021-07-19 06:07 UTC (permalink / raw)
To: torikoshia <[email protected]>; Masahiro Ikeda <[email protected]>; +Cc: Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021/07/19 11:28, torikoshia wrote:
> Agreed. Updated the patch.
Thanks for updating the patch!
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
I don't think that procsignal.c is proper place to check the permission and
check whether the specified PID indicates a PostgreSQL server process, etc
because procsignal.c just provides fundamental routines for interprocess
signaling. Isn't it better to move the function to signalfuncs.c or elsewhere?
+ ExplainQueryText(es, ActivePortal->queryDesc);
+ ExplainPrintPlan(es, ActivePortal->queryDesc);
+ ExplainPrintJITSummary(es, ActivePortal->queryDesc);
When text format is used, ExplainBeginOutput() and ExplainEndOutput()
do nothing. So (I guess) you thought that they don't need to be called and
implemented the code in that way. But IMO it's better to comment
why they don't need to be called, or to just call both of them
even if they do nothing in text format.
+ ExplainPrintJITSummary(es, ActivePortal->queryDesc);
It's better to check es->costs before calling this function,
like explain_ExecutorEnd() and ExplainOnePlan() do?
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
Currently SendProcSignalForLogInfo() calls PG_RETURN_BOOL() in some cases,
but instead it should just return true/false because pg_log_current_query_plan()
expects that?
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-07-27 18:34 ` Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Fujii Masao @ 2021-07-27 18:34 UTC (permalink / raw)
To: torikoshia <[email protected]>; Bharath Rupireddy <[email protected]>; +Cc: Dilip Kumar <[email protected]>; pgsql-hackers
On 2021/07/09 14:05, torikoshia wrote:
> On 2021-07-02 23:21, Bharath Rupireddy wrote:
>> On Tue, Jun 22, 2021 at 8:00 AM torikoshia <[email protected]> wrote:
>>> Updated the patch.
>>
>> Thanks for the patch. Here are some comments on the v4 patch:
>
> Thanks for your comments and suggestions!
> I agree with you and updated the patch.
>
> On Thu, Jul 1, 2021 at 3:34 PM Fujii Masao <[email protected]> wrote:
>
>> DO $$
>> BEGIN
>> PERFORM pg_sleep(100);
>> END$$;
>>
>> When I called pg_log_current_query_plan() to send the signal to
>> the backend executing the above query, I got the following log message.
>> I think that this is not expected message. I guess this issue happened
>> because the information about query text and plan is retrieved
>> from ActivePortal. If this understanding is right, ISTM that we should
>> implement new mechanism so that we can retrieve those information
>> even while nested query is being executed.
>
> I'm now working on this comment.
One idea is to define new global pointer, e.g., "QueryDesc *ActiveQueryDesc;".
This global pointer is set to queryDesc in ExecutorRun()
(also maybe ExecutorStart()). And this is reset to NULL in ExecutorEnd() and
when an error is thrown. Then ProcessLogCurrentPlanInterrupt() can
get the plan of the currently running query from that global pointer
instead of ActivePortal, and log it. Thought?
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
@ 2021-07-27 18:45 ` Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Pavel Stehule @ 2021-07-27 18:45 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: torikoshia <[email protected]>; Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
út 27. 7. 2021 v 20:34 odesílatel Fujii Masao <[email protected]>
napsal:
>
>
> On 2021/07/09 14:05, torikoshia wrote:
> > On 2021-07-02 23:21, Bharath Rupireddy wrote:
> >> On Tue, Jun 22, 2021 at 8:00 AM torikoshia <[email protected]>
> wrote:
> >>> Updated the patch.
> >>
> >> Thanks for the patch. Here are some comments on the v4 patch:
> >
> > Thanks for your comments and suggestions!
> > I agree with you and updated the patch.
> >
> > On Thu, Jul 1, 2021 at 3:34 PM Fujii Masao <[email protected]>
> wrote:
> >
> >> DO $$
> >> BEGIN
> >> PERFORM pg_sleep(100);
> >> END$$;
> >>
> >> When I called pg_log_current_query_plan() to send the signal to
> >> the backend executing the above query, I got the following log message.
> >> I think that this is not expected message. I guess this issue happened
> >> because the information about query text and plan is retrieved
> >> from ActivePortal. If this understanding is right, ISTM that we should
> >> implement new mechanism so that we can retrieve those information
> >> even while nested query is being executed.
> >
> > I'm now working on this comment.
>
> One idea is to define new global pointer, e.g., "QueryDesc
> *ActiveQueryDesc;".
> This global pointer is set to queryDesc in ExecutorRun()
> (also maybe ExecutorStart()). And this is reset to NULL in ExecutorEnd()
> and
> when an error is thrown. Then ProcessLogCurrentPlanInterrupt() can
> get the plan of the currently running query from that global pointer
> instead of ActivePortal, and log it. Thought?
>
It cannot work - there can be a lot of nested queries, and at the end you
cannot reset to null, but you should return back pointer to outer query.
Regards
Pavel
> Regards,
>
> --
> Fujii Masao
> Advanced Computing Technology Center
> Research and Development Headquarters
> NTT DATA CORPORATION
>
>
>
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
@ 2021-07-28 11:44 ` torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-07-28 11:44 UTC (permalink / raw)
To: Pavel Stehule <[email protected]>; +Cc: Fujii Masao <[email protected]>; Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-07-28 03:45, Pavel Stehule wrote:
> út 27. 7. 2021 v 20:34 odesílatel Fujii Masao
> <[email protected]> napsal:
>
>> On 2021/07/09 14:05, torikoshia wrote:
>>> On 2021-07-02 23:21, Bharath Rupireddy wrote:
>>>> On Tue, Jun 22, 2021 at 8:00 AM torikoshia
>> <[email protected]> wrote:
>>>>> Updated the patch.
>>>>
>>>> Thanks for the patch. Here are some comments on the v4 patch:
>>>
>>> Thanks for your comments and suggestions!
>>> I agree with you and updated the patch.
>>>
>>> On Thu, Jul 1, 2021 at 3:34 PM Fujii Masao
>> <[email protected]> wrote:
>>>
>>>> DO $$
>>>> BEGIN
>>>> PERFORM pg_sleep(100);
>>>> END$$;
>>>>
>>>> When I called pg_log_current_query_plan() to send the signal to
>>>> the backend executing the above query, I got the following log
>> message.
>>>> I think that this is not expected message. I guess this issue
>> happened
>>>> because the information about query text and plan is retrieved
>>>> from ActivePortal. If this understanding is right, ISTM that we
>> should
>>>> implement new mechanism so that we can retrieve those information
>>>> even while nested query is being executed.
>>>
>>> I'm now working on this comment.
>>
>> One idea is to define new global pointer, e.g., "QueryDesc
>> *ActiveQueryDesc;".
>> This global pointer is set to queryDesc in ExecutorRun()
>> (also maybe ExecutorStart()). And this is reset to NULL in
>> ExecutorEnd() and
>> when an error is thrown. Then ProcessLogCurrentPlanInterrupt() can
>> get the plan of the currently running query from that global pointer
>> instead of ActivePortal, and log it. Thought?
>
> It cannot work - there can be a lot of nested queries, and at the end
> you cannot reset to null, but you should return back pointer to outer
> query.
Thanks for your comment!
I'm wondering if we can avoid this problem by saving one outer level
QueryDesc in addition to the current one.
I'm going to try it.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-08-10 12:22 ` torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-08-10 12:22 UTC (permalink / raw)
To: Pavel Stehule <[email protected]>; +Cc: Fujii Masao <[email protected]>; Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-07-28 20:44, torikoshia wrote:
> On 2021-07-28 03:45, Pavel Stehule wrote:
>> út 27. 7. 2021 v 20:34 odesílatel Fujii Masao
>> <[email protected]> napsal:
>>
>>> On 2021/07/09 14:05, torikoshia wrote:
>>>> On 2021-07-02 23:21, Bharath Rupireddy wrote:
>>>>> On Tue, Jun 22, 2021 at 8:00 AM torikoshia
>>> <[email protected]> wrote:
>>>>>> Updated the patch.
>>>>>
>>>>> Thanks for the patch. Here are some comments on the v4 patch:
>>>>
>>>> Thanks for your comments and suggestions!
>>>> I agree with you and updated the patch.
>>>>
>>>> On Thu, Jul 1, 2021 at 3:34 PM Fujii Masao
>>> <[email protected]> wrote:
>>>>
>>>>> DO $$
>>>>> BEGIN
>>>>> PERFORM pg_sleep(100);
>>>>> END$$;
>>>>>
>>>>> When I called pg_log_current_query_plan() to send the signal to
>>>>> the backend executing the above query, I got the following log
>>> message.
>>>>> I think that this is not expected message. I guess this issue
>>> happened
>>>>> because the information about query text and plan is retrieved
>>>>> from ActivePortal. If this understanding is right, ISTM that we
>>> should
>>>>> implement new mechanism so that we can retrieve those information
>>>>> even while nested query is being executed.
>>>>
>>>> I'm now working on this comment.
>>>
>>> One idea is to define new global pointer, e.g., "QueryDesc
>>> *ActiveQueryDesc;".
>>> This global pointer is set to queryDesc in ExecutorRun()
>>> (also maybe ExecutorStart()). And this is reset to NULL in
>>> ExecutorEnd() and
>>> when an error is thrown. Then ProcessLogCurrentPlanInterrupt() can
>>> get the plan of the currently running query from that global pointer
>>> instead of ActivePortal, and log it. Thought?
>>
>> It cannot work - there can be a lot of nested queries, and at the end
>> you cannot reset to null, but you should return back pointer to outer
>> query.
>
> Thanks for your comment!
>
> I'm wondering if we can avoid this problem by saving one outer level
> QueryDesc in addition to the current one.
> I'm going to try it.
I have updated the patch in this way.
In this patch, getting the plan to the DO statement is as follows.
---------------------------------
(pid:76608)=# DO $$
BEGIN
PERFORM pg_sleep(15);
END$$;
(pid:74482)=# SELECT pg_log_current_query_plan(76608);
LOG: 00000: plan of the query running on backend with PID 76608 is:
Query Text: SELECT pg_sleep(15)
Result (cost=0.00..0.01 rows=1 width=4)
Output: pg_sleep('15'::double precision)
-- pid:76608 finished DO statement:
(pid:74482)=# SELECT pg_log_current_query_plan(76608);
LOG: 00000: backend with PID 76608 is not running a query
---------------------------------
Any thoughts?
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v7-0001-log-running-query-plan.patch (22.1K, ../../[email protected]/2-v7-0001-log-running-query-plan.patch)
download | inline diff:
From 74b6cdd70de2c6ed0f4c8370af892584ad9d9a4f Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 10 Aug 2021 15:38:57 +0900
Subject: [PATCH v7] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_current_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda
---
doc/src/sgml/func.sgml | 46 +++++++++++
src/backend/commands/explain.c | 83 ++++++++++++++++++++
src/backend/executor/execMain.c | 10 +++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 61 ++++++++++++++
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 54 ++-----------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 ++
src/include/commands/explain.h | 2 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 ++++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 16 +++-
src/test/regress/sql/misc_functions.sql | 12 +--
16 files changed, 270 insertions(+), 57 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 78812b2dbe..13b44b42cb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25281,6 +25281,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25394,6 +25415,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the deepest nesting query's plan is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..71538c1b6b 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -4922,3 +4926,82 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..9b9653544a 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..1aad120791 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,63 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 58b5960e27..8b8808db8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4278,6 +4282,9 @@ PostgresMain(int argc, char *argv[],
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..304f36bda0 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -161,57 +161,15 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid;
+ bool result;
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b603700ed9..0321726227 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..fa81beb553 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -124,4 +124,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 2e2e9a364a..097ebaa96d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..98017f83a9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,26 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..979a418e8e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,17 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: 152c2e0ae1a8d0ed810b2e833b536e64b91da0a6
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-08-10 15:21 ` Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Fujii Masao @ 2021-08-10 15:21 UTC (permalink / raw)
To: torikoshia <[email protected]>; Pavel Stehule <[email protected]>; +Cc: Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021/08/10 21:22, torikoshia wrote:
> I have updated the patch in this way.
Thanks for updating the patch!
> In this patch, getting the plan to the DO statement is as follows.
Looks good to me.
> Any thoughts?
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true)));
Shouldn't we hide context information by calling errhidecontext(true)?
While "make installcheck" regression test was running, I repeated
executing pg_log_current_query_plan() and got the failure of join_hash test
with the following diff. This means that pg_log_current_query_plan() could
cause the query that should be completed successfully to fail with the error.
Isn't this a bug?
I *guess* that the cause of this issue is that ExplainNode() can call
InstrEndLoop() more than once unexpectedly.
------------------------------------------------------------------------------
$$
select count(*) from simple r join simple s using (id);
$$);
- initially_multibatch | increased_batches
-----------------------+-------------------
- f | f
-(1 row)
-
+ERROR: InstrEndLoop called on running node
+CONTEXT: PL/pgSQL function hash_join_batches(text) line 6 at FOR over EXECUTE statement
rollback to settings;
-- parallel with parallel-oblivious hash join
savepoint settings;
@@ -687,11 +684,9 @@
left join (select b1.id, b1.t from join_bar b1 join join_bar b2 using (id)) ss
on join_foo.id < ss.id + 1 and join_foo.id > ss.id - 1;
$$);
- multibatch
-------------
- t
-(1 row)
-
+ERROR: InstrEndLoop called on running node
+CONTEXT: parallel worker
+PL/pgSQL function hash_join_batches(text) line 6 at FOR over EXECUTE statement
rollback to settings;
-- single-batch with rescan, parallel-aware
savepoint settings;
------------------------------------------------------------------------------
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
@ 2021-08-11 12:14 ` torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-08-11 12:14 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: Pavel Stehule <[email protected]>; Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-08-11 00:21, Fujii Masao wrote:
> On 2021/08/10 21:22, torikoshia wrote:
>> I have updated the patch in this way.
>
> Thanks for updating the patch!
>
>
>> In this patch, getting the plan to the DO statement is as follows.
>
> Looks good to me.
>
>
>> Any thoughts?
>
> + ereport(LOG_SERVER_ONLY,
> + (errmsg("plan of the query running on backend with PID %d is:\n%s",
> + MyProcPid, es->str->data),
> + errhidestmt(true)));
>
> Shouldn't we hide context information by calling errhidecontext(true)?
Agreed.
> While "make installcheck" regression test was running, I repeated
> executing pg_log_current_query_plan() and got the failure of join_hash
> test
> with the following diff. This means that pg_log_current_query_plan()
> could
> cause the query that should be completed successfully to fail with the
> error.
> Isn't this a bug?
Thanks for finding the bug.
I also reproduced it.
> I *guess* that the cause of this issue is that ExplainNode() can call
> InstrEndLoop() more than once unexpectedly.
As far as I looked into, pg_log_current_plan() can call InstrEndLoop()
through ExplainNode().
I added a flag to ExplainState to avoid calling InstrEndLoop() when
ExplainNode() is called from pg_log_current_plan().
>
>
> ------------------------------------------------------------------------------
> $$
> select count(*) from simple r join simple s using (id);
> $$);
> - initially_multibatch | increased_batches
> -----------------------+-------------------
> - f | f
> -(1 row)
> -
> +ERROR: InstrEndLoop called on running node
> +CONTEXT: PL/pgSQL function hash_join_batches(text) line 6 at FOR
> over EXECUTE statement
> rollback to settings;
> -- parallel with parallel-oblivious hash join
> savepoint settings;
> @@ -687,11 +684,9 @@
> left join (select b1.id, b1.t from join_bar b1 join join_bar b2
> using (id)) ss
> on join_foo.id < ss.id + 1 and join_foo.id > ss.id - 1;
> $$);
> - multibatch
> -------------
> - t
> -(1 row)
> -
> +ERROR: InstrEndLoop called on running node
> +CONTEXT: parallel worker
> +PL/pgSQL function hash_join_batches(text) line 6 at FOR over EXECUTE
> statement
> rollback to settings;
> -- single-batch with rescan, parallel-aware
> savepoint settings;
>
> ------------------------------------------------------------------------------
>
> Regards,
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v8-0001-log-running-query-plan.patch (23.3K, ../../[email protected]/2-v8-0001-log-running-query-plan.patch)
download | inline diff:
From 26356efb094ac25b11fe65df93f11c64ad136723 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 11 Aug 2021 20:17:42 +0900
Subject: [PATCH v8] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_current_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda
---
doc/src/sgml/func.sgml | 46 ++++++++++
src/backend/commands/explain.c | 90 +++++++++++++++++++-
src/backend/executor/execMain.c | 10 +++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 61 +++++++++++++
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 54 ++----------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 ++
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 +++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 16 +++-
src/test/regress/sql/misc_functions.sql | 12 +--
16 files changed, 277 insertions(+), 58 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 78812b2dbe..13b44b42cb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25281,6 +25281,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25394,6 +25415,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the deepest nesting query's plan is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..75524eb1b8 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -1594,6 +1598,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1608,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4922,3 +4929,84 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..9b9653544a 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..1aad120791 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,63 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 58b5960e27..8b8808db8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4278,6 +4282,9 @@ PostgresMain(int argc, char *argv[],
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..304f36bda0 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -161,57 +161,15 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid;
+ bool result;
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b603700ed9..0321726227 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 2e2e9a364a..097ebaa96d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..98017f83a9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,26 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..979a418e8e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,17 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: a6bd28beb0639d4cf424e961862a65c466ca65bf
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-08-19 16:12 ` Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Fujii Masao @ 2021-08-19 16:12 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Pavel Stehule <[email protected]>; Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021/08/11 21:14, torikoshia wrote:
> As far as I looked into, pg_log_current_plan() can call InstrEndLoop() through ExplainNode().
> I added a flag to ExplainState to avoid calling InstrEndLoop() when ExplainNode() is called from pg_log_current_plan().
Thanks for updating the patch!
I tried to test the patch again and encountered two issues.
(1)
The following WITH RECURSIVE query failed with the error
"ERROR: failed to find plan for CTE sg" when I ran
pg_log_current_query_plan() against the backend executing that query.
Is this a bug?
create table graph0( f int, t int, label text );
insert into graph0 values (1, 2, 'arc 1 -> 2'),(1, 3, 'arc 1 -> 3'),(2, 3, 'arc 2 -> 3'),(1, 4, 'arc 1 -> 4'),(4, 5, 'arc 4 -> 5');
with recursive search_graph(f, t, label, i) as (
select *, 1||pg_sleep(1)::text from graph0 g
union distinct
select g.*,1||pg_sleep(1)::text
from graph0 g, search_graph sg
where g.f = sg.t
) search breadth first by f, t set seq
select * from search_graph order by seq;
(2)
When I ran pg_log_current_query_plan() while "make installcheck" test
was running, I got the following assertion failure.
TRAP: FailedAssertion("!IsPageLockHeld || (locktag->locktag_type == LOCKTAG_RELATION_EXTEND)", File: "lock.c", Line: 894, PID: 61512)
0 postgres 0x000000010ec23557 ExceptionalCondition + 231
1 postgres 0x000000010e9eff15 LockAcquireExtended + 1461
2 postgres 0x000000010e9ed14d LockRelationOid + 61
3 postgres 0x000000010e41251b relation_open + 91
4 postgres 0x000000010e509679 table_open + 25
5 postgres 0x000000010ebf9462 SearchCatCacheMiss + 274
6 postgres 0x000000010ebf5979 SearchCatCacheInternal + 761
7 postgres 0x000000010ebf566c SearchCatCache + 60
8 postgres 0x000000010ec1a9e0 SearchSysCache + 144
9 postgres 0x000000010ec1ae03 SearchSysCacheExists + 51
10 postgres 0x000000010e58ce35 TypeIsVisible + 437
11 postgres 0x000000010ea98e4c format_type_extended + 1964
12 postgres 0x000000010ea9900e format_type_with_typemod + 30
13 postgres 0x000000010eb78d76 get_const_expr + 742
14 postgres 0x000000010eb79bc8 get_rule_expr + 232
15 postgres 0x000000010eb8140f get_func_expr + 1247
16 postgres 0x000000010eb79dcd get_rule_expr + 749
17 postgres 0x000000010eb81688 get_rule_expr_paren + 136
18 postgres 0x000000010eb7bf38 get_rule_expr + 9304
19 postgres 0x000000010eb72ad5 deparse_expression_pretty + 149
20 postgres 0x000000010eb73463 deparse_expression + 83
21 postgres 0x000000010e68eaf1 show_plan_tlist + 353
22 postgres 0x000000010e68adaf ExplainNode + 4991
23 postgres 0x000000010e688b4b ExplainPrintPlan + 283
24 postgres 0x000000010e68e1aa ProcessLogCurrentPlanInterrupt + 266
25 postgres 0x000000010ea133bb ProcessInterrupts + 3435
26 postgres 0x000000010e738c97 vacuum_delay_point + 55
27 postgres 0x000000010e42bb4b ginInsertCleanup + 1531
28 postgres 0x000000010e42d418 gin_clean_pending_list + 776
29 postgres 0x000000010e74955a ExecInterpExpr + 2522
30 postgres 0x000000010e7487e2 ExecInterpExprStillValid + 82
31 postgres 0x000000010e7ae83b ExecEvalExprSwitchContext + 59
32 postgres 0x000000010e7ae7be ExecProject + 78
33 postgres 0x000000010e7ae4e9 ExecResult + 345
34 postgres 0x000000010e764e42 ExecProcNodeFirst + 82
35 postgres 0x000000010e75ccb2 ExecProcNode + 50
36 postgres 0x000000010e758301 ExecutePlan + 193
37 postgres 0x000000010e7581d1 standard_ExecutorRun + 609
38 auto_explain.so 0x000000010f1df3a7 explain_ExecutorRun + 247
39 postgres 0x000000010e757f3b ExecutorRun + 91
40 postgres 0x000000010ea1cb49 PortalRunSelect + 313
41 postgres 0x000000010ea1c4dd PortalRun + 861
42 postgres 0x000000010ea17474 exec_simple_query + 1540
43 postgres 0x000000010ea164d4 PostgresMain + 2580
44 postgres 0x000000010e91d159 BackendRun + 89
45 postgres 0x000000010e91c6a5 BackendStartup + 565
46 postgres 0x000000010e91b3fe ServerLoop + 638
47 postgres 0x000000010e918b9d PostmasterMain + 6717
48 postgres 0x000000010e7efd43 main + 819
49 libdyld.dylib 0x00007fff6a46e3d5 start + 1
50 ??? 0x0000000000000003 0x0 + 3
LOG: server process (PID 61512) was terminated by signal 6: Abort trap: 6
DETAIL: Failed process was running: select gin_clean_pending_list('t_gin_test_tbl_i_j_idx') is not null;
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
@ 2021-09-07 03:39 ` torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-09-07 03:39 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: Pavel Stehule <[email protected]>; Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-08-20 01:12, Fujii Masao wrote:
> On 2021/08/11 21:14, torikoshia wrote:
>> As far as I looked into, pg_log_current_plan() can call InstrEndLoop()
>> through ExplainNode().
>> I added a flag to ExplainState to avoid calling InstrEndLoop() when
>> ExplainNode() is called from pg_log_current_plan().
>
> Thanks for updating the patch!
> I tried to test the patch again and encountered two issues.
Thanks for finding these issues!
>
> (1)
> The following WITH RECURSIVE query failed with the error
> "ERROR: failed to find plan for CTE sg" when I ran
> pg_log_current_query_plan() against the backend executing that query.
> Is this a bug?
>
> create table graph0( f int, t int, label text );
> insert into graph0 values (1, 2, 'arc 1 -> 2'),(1, 3, 'arc 1 ->
> 3'),(2, 3, 'arc 2 -> 3'),(1, 4, 'arc 1 -> 4'),(4, 5, 'arc 4 -> 5');
>
> with recursive search_graph(f, t, label, i) as (
> select *, 1||pg_sleep(1)::text from graph0 g
> union distinct
> select g.*,1||pg_sleep(1)::text
> from graph0 g, search_graph sg
> where g.f = sg.t
> ) search breadth first by f, t set seq
> select * from search_graph order by seq;
This ERROR occurred without applying the patch and just calling
EXPLAIN(VERBOSE) to CTE with SEARCH BREADTH FIRST.
I'm going to make another thread to discuss it.
> (2)
> When I ran pg_log_current_query_plan() while "make installcheck" test
> was running, I got the following assertion failure.
>
> TRAP: FailedAssertion("!IsPageLockHeld || (locktag->locktag_type ==
> LOCKTAG_RELATION_EXTEND)", File: "lock.c", Line: 894, PID: 61512)
>
> 0 postgres 0x000000010ec23557
> ExceptionalCondition + 231
> 1 postgres 0x000000010e9eff15
> LockAcquireExtended + 1461
> 2 postgres 0x000000010e9ed14d
> LockRelationOid + 61
> 3 postgres 0x000000010e41251b
> relation_open + 91
> 4 postgres 0x000000010e509679 table_open +
> 25
> 5 postgres 0x000000010ebf9462
> SearchCatCacheMiss + 274
> 6 postgres 0x000000010ebf5979
> SearchCatCacheInternal + 761
> 7 postgres 0x000000010ebf566c
> SearchCatCache + 60
> 8 postgres 0x000000010ec1a9e0
> SearchSysCache + 144
> 9 postgres 0x000000010ec1ae03
> SearchSysCacheExists + 51
> 10 postgres 0x000000010e58ce35
> TypeIsVisible + 437
> 11 postgres 0x000000010ea98e4c
> format_type_extended + 1964
> 12 postgres 0x000000010ea9900e
> format_type_with_typemod + 30
> 13 postgres 0x000000010eb78d76
> get_const_expr + 742
> 14 postgres 0x000000010eb79bc8
> get_rule_expr + 232
> 15 postgres 0x000000010eb8140f
> get_func_expr + 1247
> 16 postgres 0x000000010eb79dcd
> get_rule_expr + 749
> 17 postgres 0x000000010eb81688
> get_rule_expr_paren + 136
> 18 postgres 0x000000010eb7bf38
> get_rule_expr + 9304
> 19 postgres 0x000000010eb72ad5
> deparse_expression_pretty + 149
> 20 postgres 0x000000010eb73463
> deparse_expression + 83
> 21 postgres 0x000000010e68eaf1
> show_plan_tlist + 353
> 22 postgres 0x000000010e68adaf ExplainNode
> + 4991
> 23 postgres 0x000000010e688b4b
> ExplainPrintPlan + 283
> 24 postgres 0x000000010e68e1aa
> ProcessLogCurrentPlanInterrupt + 266
> 25 postgres 0x000000010ea133bb
> ProcessInterrupts + 3435
> 26 postgres 0x000000010e738c97
> vacuum_delay_point + 55
> 27 postgres 0x000000010e42bb4b
> ginInsertCleanup + 1531
> 28 postgres 0x000000010e42d418
> gin_clean_pending_list + 776
> 29 postgres 0x000000010e74955a
> ExecInterpExpr + 2522
> 30 postgres 0x000000010e7487e2
> ExecInterpExprStillValid + 82
> 31 postgres 0x000000010e7ae83b
> ExecEvalExprSwitchContext + 59
> 32 postgres 0x000000010e7ae7be ExecProject
> + 78
> 33 postgres 0x000000010e7ae4e9 ExecResult +
> 345
> 34 postgres 0x000000010e764e42
> ExecProcNodeFirst + 82
> 35 postgres 0x000000010e75ccb2 ExecProcNode
> + 50
> 36 postgres 0x000000010e758301 ExecutePlan
> + 193
> 37 postgres 0x000000010e7581d1
> standard_ExecutorRun + 609
> 38 auto_explain.so 0x000000010f1df3a7
> explain_ExecutorRun + 247
> 39 postgres 0x000000010e757f3b ExecutorRun
> + 91
> 40 postgres 0x000000010ea1cb49
> PortalRunSelect + 313
> 41 postgres 0x000000010ea1c4dd PortalRun +
> 861
> 42 postgres 0x000000010ea17474
> exec_simple_query + 1540
> 43 postgres 0x000000010ea164d4 PostgresMain
> + 2580
> 44 postgres 0x000000010e91d159 BackendRun +
> 89
> 45 postgres 0x000000010e91c6a5
> BackendStartup + 565
> 46 postgres 0x000000010e91b3fe ServerLoop +
> 638
> 47 postgres 0x000000010e918b9d
> PostmasterMain + 6717
> 48 postgres 0x000000010e7efd43 main + 819
> 49 libdyld.dylib 0x00007fff6a46e3d5 start + 1
> 50 ??? 0x0000000000000003 0x0 + 3
>
> LOG: server process (PID 61512) was terminated by signal 6: Abort
> trap: 6
> DETAIL: Failed process was running: select
> gin_clean_pending_list('t_gin_test_tbl_i_j_idx') is not null;
As far as I understand, since explaining plans can acquire heavyweight
lock for example to get column names, when page lock is held at the time
of the interrupt, this assertion error occurs.
The attached patch tries to avoid this by checking each LocalLock entry
and when finding even one, giving up logging the plan.
Thoughts?
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v9-0001-log-running-query-plan.patch (24.3K, ../../[email protected]/2-v9-0001-log-running-query-plan.patch)
download | inline diff:
From 26356efb094ac25b11fe65df93f11c64ad136723 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 6 Sep 2021 14:18:51 +0900
Subject: [PATCH v8] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_current_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda
---
doc/src/sgml/func.sgml | 46 ++++++++
src/backend/commands/explain.c | 117 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 61 ++++++++++
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 54 +--------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 ++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 16 ++-
src/test/regress/sql/misc_functions.sql | 12 +-
16 files changed, 303 insertions(+), 59 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 78812b2dbe..13b44b42cb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25281,6 +25281,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25394,6 +25415,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the deepest nesting query's plan is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..f528301cc5 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +44,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1597,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1607,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4922,3 +4928,110 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is now holding a page lock. Try again",
+ MyProcPid)));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..9b9653544a 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..1aad120791 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,63 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 58b5960e27..8b8808db8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4278,6 +4282,9 @@ PostgresMain(int argc, char *argv[],
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..304f36bda0 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -161,57 +161,15 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid;
+ bool result;
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c07b2c6a55..6b4ef16d81 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 2e2e9a364a..097ebaa96d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..98017f83a9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,26 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..979a418e8e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,17 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: 0bd305ee1d427ef29f5fa4fa20567e3b3f5ff792
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-09-08 12:06 ` torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-09-08 12:06 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: Pavel Stehule <[email protected]>; Bharath Rupireddy <[email protected]>; Dilip Kumar <[email protected]>; pgsql-hackers
On 2021-09-07 12:39, torikoshia wrote:
> On 2021-08-20 01:12, Fujii Masao wrote:
>> On 2021/08/11 21:14, torikoshia wrote:
>>> As far as I looked into, pg_log_current_plan() can call
>>> InstrEndLoop() through ExplainNode().
>>> I added a flag to ExplainState to avoid calling InstrEndLoop() when
>>> ExplainNode() is called from pg_log_current_plan().
>>
>> Thanks for updating the patch!
>> I tried to test the patch again and encountered two issues.
>
> Thanks for finding these issues!
>
>>
>> (1)
>> The following WITH RECURSIVE query failed with the error
>> "ERROR: failed to find plan for CTE sg" when I ran
>> pg_log_current_query_plan() against the backend executing that query.
>> Is this a bug?
>>
>> create table graph0( f int, t int, label text );
>> insert into graph0 values (1, 2, 'arc 1 -> 2'),(1, 3, 'arc 1 ->
>> 3'),(2, 3, 'arc 2 -> 3'),(1, 4, 'arc 1 -> 4'),(4, 5, 'arc 4 -> 5');
>>
>> with recursive search_graph(f, t, label, i) as (
>> select *, 1||pg_sleep(1)::text from graph0 g
>> union distinct
>> select g.*,1||pg_sleep(1)::text
>> from graph0 g, search_graph sg
>> where g.f = sg.t
>> ) search breadth first by f, t set seq
>> select * from search_graph order by seq;
>
> This ERROR occurred without applying the patch and just calling
> EXPLAIN(VERBOSE) to CTE with SEARCH BREADTH FIRST.
>
> I'm going to make another thread to discuss it.
>
>> (2)
>> When I ran pg_log_current_query_plan() while "make installcheck" test
>> was running, I got the following assertion failure.
>>
>> TRAP: FailedAssertion("!IsPageLockHeld || (locktag->locktag_type ==
>> LOCKTAG_RELATION_EXTEND)", File: "lock.c", Line: 894, PID: 61512)
>>
>> 0 postgres 0x000000010ec23557
>> ExceptionalCondition + 231
>> 1 postgres 0x000000010e9eff15
>> LockAcquireExtended + 1461
>> 2 postgres 0x000000010e9ed14d
>> LockRelationOid + 61
>> 3 postgres 0x000000010e41251b
>> relation_open + 91
>> 4 postgres 0x000000010e509679 table_open
>> + 25
>> 5 postgres 0x000000010ebf9462
>> SearchCatCacheMiss + 274
>> 6 postgres 0x000000010ebf5979
>> SearchCatCacheInternal + 761
>> 7 postgres 0x000000010ebf566c
>> SearchCatCache + 60
>> 8 postgres 0x000000010ec1a9e0
>> SearchSysCache + 144
>> 9 postgres 0x000000010ec1ae03
>> SearchSysCacheExists + 51
>> 10 postgres 0x000000010e58ce35
>> TypeIsVisible + 437
>> 11 postgres 0x000000010ea98e4c
>> format_type_extended + 1964
>> 12 postgres 0x000000010ea9900e
>> format_type_with_typemod + 30
>> 13 postgres 0x000000010eb78d76
>> get_const_expr + 742
>> 14 postgres 0x000000010eb79bc8
>> get_rule_expr + 232
>> 15 postgres 0x000000010eb8140f
>> get_func_expr + 1247
>> 16 postgres 0x000000010eb79dcd
>> get_rule_expr + 749
>> 17 postgres 0x000000010eb81688
>> get_rule_expr_paren + 136
>> 18 postgres 0x000000010eb7bf38
>> get_rule_expr + 9304
>> 19 postgres 0x000000010eb72ad5
>> deparse_expression_pretty + 149
>> 20 postgres 0x000000010eb73463
>> deparse_expression + 83
>> 21 postgres 0x000000010e68eaf1
>> show_plan_tlist + 353
>> 22 postgres 0x000000010e68adaf ExplainNode
>> + 4991
>> 23 postgres 0x000000010e688b4b
>> ExplainPrintPlan + 283
>> 24 postgres 0x000000010e68e1aa
>> ProcessLogCurrentPlanInterrupt + 266
>> 25 postgres 0x000000010ea133bb
>> ProcessInterrupts + 3435
>> 26 postgres 0x000000010e738c97
>> vacuum_delay_point + 55
>> 27 postgres 0x000000010e42bb4b
>> ginInsertCleanup + 1531
>> 28 postgres 0x000000010e42d418
>> gin_clean_pending_list + 776
>> 29 postgres 0x000000010e74955a
>> ExecInterpExpr + 2522
>> 30 postgres 0x000000010e7487e2
>> ExecInterpExprStillValid + 82
>> 31 postgres 0x000000010e7ae83b
>> ExecEvalExprSwitchContext + 59
>> 32 postgres 0x000000010e7ae7be ExecProject
>> + 78
>> 33 postgres 0x000000010e7ae4e9 ExecResult
>> + 345
>> 34 postgres 0x000000010e764e42
>> ExecProcNodeFirst + 82
>> 35 postgres 0x000000010e75ccb2
>> ExecProcNode + 50
>> 36 postgres 0x000000010e758301 ExecutePlan
>> + 193
>> 37 postgres 0x000000010e7581d1
>> standard_ExecutorRun + 609
>> 38 auto_explain.so 0x000000010f1df3a7
>> explain_ExecutorRun + 247
>> 39 postgres 0x000000010e757f3b ExecutorRun
>> + 91
>> 40 postgres 0x000000010ea1cb49
>> PortalRunSelect + 313
>> 41 postgres 0x000000010ea1c4dd PortalRun +
>> 861
>> 42 postgres 0x000000010ea17474
>> exec_simple_query + 1540
>> 43 postgres 0x000000010ea164d4
>> PostgresMain + 2580
>> 44 postgres 0x000000010e91d159 BackendRun
>> + 89
>> 45 postgres 0x000000010e91c6a5
>> BackendStartup + 565
>> 46 postgres 0x000000010e91b3fe ServerLoop
>> + 638
>> 47 postgres 0x000000010e918b9d
>> PostmasterMain + 6717
>> 48 postgres 0x000000010e7efd43 main + 819
>> 49 libdyld.dylib 0x00007fff6a46e3d5 start + 1
>> 50 ??? 0x0000000000000003 0x0 + 3
>>
>> LOG: server process (PID 61512) was terminated by signal 6: Abort
>> trap: 6
>> DETAIL: Failed process was running: select
>> gin_clean_pending_list('t_gin_test_tbl_i_j_idx') is not null;
>
> As far as I understand, since explaining plans can acquire heavyweight
> lock for example to get column names, when page lock is held at the
> time of the interrupt, this assertion error occurs.
>
> The attached patch tries to avoid this by checking each LocalLock
> entry and when finding even one, giving up logging the plan.
>
> Thoughts?
Regression tests failed on windows.
Updated patch attached.
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v10-0001-log-running-query-plan.patch (25.3K, ../../[email protected]/2-v10-0001-log-running-query-plan.patch)
download | inline diff:
From 26356efb094ac25b11fe65df93f11c64ad136723 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 8 Sep 2021 20:40:21 +0900
Subject: [PATCH v8] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_current_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda
---
doc/src/sgml/func.sgml | 46 ++++++++
src/backend/commands/explain.c | 117 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 61 ++++++++++
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 54 +--------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 ++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 16 ++-
src/test/regress/sql/misc_functions.sql | 12 +-
17 files changed, 308 insertions(+), 63 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 78812b2dbe..13b44b42cb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25281,6 +25281,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25394,6 +25415,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the deepest nesting query's plan is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..f528301cc5 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +44,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1597,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1607,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4922,3 +4928,110 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is now holding a page lock. Try again",
+ MyProcPid)));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..9b9653544a 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..1aad120791 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,63 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 364654e106..6d1a5add17 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 58b5960e27..8b8808db8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4278,6 +4282,9 @@ PostgresMain(int argc, char *argv[],
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..304f36bda0 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -161,57 +161,15 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid;
+ bool result;
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d068d6532e..3a81e86493 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 2e2e9a364a..097ebaa96d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..98017f83a9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,26 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..979a418e8e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,17 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: a3d2b1bbe904b0ca8d9fdde20f25295ff3e21f79
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-10-13 14:28 ` Ekaterina Sokolova <[email protected]>
2021-10-15 06:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-12 18:37 ` Re: RFC: Logging plan of the running query Justin Pryzby <[email protected]>
2021-11-13 13:29 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 3 replies; 124+ messages in thread
From: Ekaterina Sokolova @ 2021-10-13 14:28 UTC (permalink / raw)
To: [email protected]; [email protected]
Hi, hackers!
• The last version of patch is correct applied. It changes 8 files from
/src/backend, and 9 other files.
• I have 1 error and 1 warning during compilation on Mac.
explain.c:4985:25: error: implicit declaration of function
'GetLockMethodLocalHash' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
hash_seq_init(&status, GetLockMethodLocalHash());
explain.c:4985:25: warning: incompatible integer to pointer conversion
passing 'int' to parameter of type 'HTAB *' (aka 'struct HTAB *')
[-Wint-conversion]
hash_seq_init(&status, GetLockMethodLocalHash());
This error doesn't appear at my second machine with Ubuntu.
I found the reason. You delete #ifdef USE_ASSERT_CHECKING from
implementation of function GetLockMethodLocalHash(void), but this ifdef
exists around function declaration. There may be a situation, when
implementation exists without declaration, so files with using of
function produce errors. I create new version of patch with fix of this
problem.
I'm agree that seeing the details of a query is a useful feature, but I
have several doubts:
1) There are lots of changes of core's code. But not all users need this
functionality. So adding this functionality like extension seemed more
reasonable.
2) There are many tools available to monitor the status of a query. How
much do we need another one? For example:
• pg_stat_progress_* is set of views with current status of ANALYZE,
CREATE INDEX, VACUUM, CLUSTER, COPY, Base Backup. You can find it in
PostgreSQL documentation [1].
• pg_query_state is contrib with 2 patches for core (I hope someday
Community will support adding this patches to PostgreSQL). It contains
function with printing table with pid, full query text, plan and current
progress of every node like momentary EXPLAIN ANALYSE for SELECT,
UPDATE, INSERT, DELETE. So it supports every flags and formats of
EXPLAIN. You can find current version of pg_query_state on github [2].
Also I found old discussion about first version of it in Community [3].
3) Have you measured the overload of your feature? It would be really
interesting to know the changes in speed and performance.
Thank you for working on this issue. I would be glad to continue to
follow the development of this issue.
Links above:
[1] https://www.postgresql.org/docs/current/progress-reporting.html
[2] https://github.com/postgrespro/pg_query_state
[3]
https://www.postgresql.org/message-id/[email protected]
--
Ekaterina Sokolova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
[text/x-diff] v11-0001-log-running-query-plan.patch (26.1K, ../../[email protected]/2-v11-0001-log-running-query-plan.patch)
download | inline diff:
From 26356efb094ac25b11fe65df93f11c64ad136723 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 8 Sep 2021 20:40:21 +0900
Subject: [PATCH v8] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
before we check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
Only superusers are allowed to request to log the plans because
allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial
of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_current_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda, Ekaterina Sokolova
---
doc/src/sgml/func.sgml | 46 ++++++++
src/backend/commands/explain.c | 117 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 61 ++++++++++
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 54 +--------
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 ++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 16 ++-
src/test/regress/sql/misc_functions.sql | 12 +-
18 files changed, 308 insertions(+), 65 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 78812b2dbe..13b44b42cb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25281,6 +25281,27 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ Only superusers can request to log plan of the running query.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25394,6 +25415,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the deepest nesting query's plan is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..f528301cc5 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +44,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1597,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1607,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4922,3 +4928,110 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is now holding a page lock. Try again",
+ MyProcPid)));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ pid_t pid;
+ bool result;
+
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN);
+
+ PG_RETURN_BOOL(result);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index defb75aa26..9b9653544a 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..1aad120791 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,63 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * Only superusers are allowed to signal to log information because
+ * allowing any users to issue this request at an unbounded rate
+ * would cause lots of log messages and which can lead to denial of
+ * service.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ /* Only allow superusers to log information. */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log information about specified process")));
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 364654e106..6d1a5add17 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 58b5960e27..8b8808db8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4278,6 +4282,9 @@ PostgresMain(int argc, char *argv[],
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 0d52613bc3..304f36bda0 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -161,57 +161,15 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
- *
- * Only superusers are allowed to signal to log the memory contexts
- * because allowing any users to issue this request at an unbounded
- * rate would cause lots of log messages and which can lead to
- * denial of service.
- *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
- int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- /* Only allow superusers to log memory contexts. */
- if (!superuser())
- ereport(ERROR,
- (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- errmsg("must be a superuser to log memory contexts")));
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid_t pid;
+ bool result;
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
+ pid = PG_GETARG_INT32(0);
+ result = SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT);
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(result);
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d068d6532e..3a81e86493 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4544', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 2e2e9a364a..097ebaa96d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 9b2a421c32c..5efac88b130 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -560,9 +560,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e845042d38..98017f83a9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,18 +134,26 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index a398349afc..979a418e8e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -29,15 +29,17 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
-- should fail, one or more arguments is required
SELECT num_nonnulls();
SELECT num_nulls();
-
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
--- Furthermore, their contents can vary depending on the timing. However,
+-- These functions return true if the specified backend is successfully
+-- signaled, otherwise false. Upon receiving the signal, the backend
+-- will log the information to the server log.
+-- Although their contents can vary depending on the timing,
-- we can at least verify that the code doesn't fail.
--
-SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+SELECT pg_log_current_query_plan(pg_backend_pid());
--
-- Test some built-in SRFs
base-commit: a3d2b1bbe904b0ca8d9fdde20f25295ff3e21f79
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
@ 2021-10-15 06:17 ` torikoshia <[email protected]>
2021-10-15 10:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-10-15 06:17 UTC (permalink / raw)
To: Ekaterina Sokolova <[email protected]>; +Cc: [email protected]
On 2021-10-13 23:28, Ekaterina Sokolova wrote:
> Hi, hackers!
>
> • The last version of patch is correct applied. It changes 8 files
> from /src/backend, and 9 other files.
>
> • I have 1 error and 1 warning during compilation on Mac.
>
> explain.c:4985:25: error: implicit declaration of function
> 'GetLockMethodLocalHash' is invalid in C99
> [-Werror,-Wimplicit-function-declaration]
> hash_seq_init(&status, GetLockMethodLocalHash());
> explain.c:4985:25: warning: incompatible integer to pointer conversion
> passing 'int' to parameter of type 'HTAB *' (aka 'struct HTAB *')
> [-Wint-conversion]
> hash_seq_init(&status, GetLockMethodLocalHash());
>
> This error doesn't appear at my second machine with Ubuntu.
>
> I found the reason. You delete #ifdef USE_ASSERT_CHECKING from
> implementation of function GetLockMethodLocalHash(void), but this
> ifdef exists around function declaration. There may be a situation,
> when implementation exists without declaration, so files with using of
> function produce errors. I create new version of patch with fix of
> this problem.
Thanks for fixing that!
> I'm agree that seeing the details of a query is a useful feature, but
> I have several doubts:
>
> 1) There are lots of changes of core's code. But not all users need
> this functionality. So adding this functionality like extension seemed
> more reasonable.
It would be good if we can implement this feature in an extension, but
as pg_query_state extension needs applying patches to PostgreSQL, I
think this kind of feature needs PostgreSQL core modification.
IMHO extensions which need core modification are not easy to use in
production environments..
> 2) There are many tools available to monitor the status of a query.
> How much do we need another one? For example:
> • pg_stat_progress_* is set of views with current status of
> ANALYZE, CREATE INDEX, VACUUM, CLUSTER, COPY, Base Backup. You can
> find it in PostgreSQL documentation [1].
> • pg_query_state is contrib with 2 patches for core (I hope
> someday Community will support adding this patches to PostgreSQL). It
> contains function with printing table with pid, full query text, plan
> and current progress of every node like momentary EXPLAIN ANALYSE for
> SELECT, UPDATE, INSERT, DELETE. So it supports every flags and formats
> of EXPLAIN. You can find current version of pg_query_state on github
> [2]. Also I found old discussion about first version of it in
> Community [3].
Thanks for introducing the extension!
I only took a quick look at pg_query_state, I have some questions.
pg_query_state seems using shm_mq to expose the plan information, but
there was a discussion that this kind of architecture would be tricky to
do properly [1].
Does pg_query_state handle difficulties listed on the discussion?
It seems the caller of the pg_query_state() has to wait until the target
process pushes the plan information into shared memory, can it lead to
deadlock situations?
I came up with this question because when trying to make a view for
memory contexts of other backends, we encountered deadlock situations.
After all, we gave up view design and adopted sending signal and
logging.
Some of the comments of [3] seem useful for my patch, I'm going to
consider them. Thanks!
> 3) Have you measured the overload of your feature? It would be really
> interesting to know the changes in speed and performance.
I haven't measured it yet, but I believe that the overhead for backends
which are not called pg_log_current_plan() would be slight since the
patch just adds the logic for saving QueryDesc on ExecutorRun().
The overhead for backends which is called pg_log_current_plan() might
not slight, but since the target process are assumed dealing with
long-running query and the user want to know its plan, its overhead
would be worth the cost.
> Thank you for working on this issue. I would be glad to continue to
> follow the development of this issue.
Thanks for your help!
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-10-15 06:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-10-15 10:12 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2021-10-15 10:12 UTC (permalink / raw)
To: Ekaterina Sokolova <[email protected]>; +Cc: [email protected]
On 2021-10-15 15:17, torikoshia wrote:
> I only took a quick look at pg_query_state, I have some questions.
>
> pg_query_state seems using shm_mq to expose the plan information, but
> there was a discussion that this kind of architecture would be tricky
> to do properly [1].
> Does pg_query_state handle difficulties listed on the discussion?
Sorry, I forgot to add the URL.
[1]
https://www.postgresql.org/message-id/9a50371e15e741e295accabc72a41df1%40oss.nttdata.com
> It seems the caller of the pg_query_state() has to wait until the
> target process pushes the plan information into shared memory, can it
> lead to deadlock situations?
> I came up with this question because when trying to make a view for
> memory contexts of other backends, we encountered deadlock situations.
> After all, we gave up view design and adopted sending signal and
> logging.
Discussion at the following URL.
https://www.postgresql.org/message-id/9a50371e15e741e295accabc72a41df1%40oss.nttdata.com
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
@ 2021-11-12 18:37 ` Justin Pryzby <[email protected]>
2021-11-15 14:00 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2 siblings, 1 reply; 124+ messages in thread
From: Justin Pryzby @ 2021-11-12 18:37 UTC (permalink / raw)
To: Ekaterina Sokolova <[email protected]>; +Cc: [email protected]; [email protected]; Fujii Masao <[email protected]>
On Wed, Oct 13, 2021 at 05:28:30PM +0300, Ekaterina Sokolova wrote:
> Hi, hackers!
>
> • pg_query_state is contrib with 2 patches for core (I hope someday
> Community will support adding this patches to PostgreSQL). It contains
I reviewed this version of the patch - I have some language fixes.
I didn't know about pg_query_state, thanks.
> To improve this situation, this patch adds
> pg_log_current_query_plan() function that requests to log the
> plan of the specified backend process.
To me, "current plan" seems to mean "plan of *this* backend" (which makes no
sense to log). I think the user-facing function could be called
pg_log_query_plan(). It's true that the implementation is a request to another
backend to log its *own* query plan - but users shouldn't need to know about
the implementation.
> + Only superusers can request to log plan of the running query.
.. log the plan of a running query.
> + Note that nested statements (statements executed inside a function) are not
> + considered for logging. Only the deepest nesting query's plan is logged.
Only the plan of the most deeply nested query is logged.
> + (errmsg("backend with PID %d is not running a query",
> + MyProcPid)));
The extra parens around errmsg() are not needed since e3a87b499.
> + (errmsg("backend with PID %d is now holding a page lock. Try again",
remove "now"
> + (errmsg("plan of the query running on backend with PID %d is:\n%s",
> + MyProcPid, es->str->data),
Maybe this should say "query plan running on backend with PID 17793 is:"
> + * would cause lots of log messages and which can lead to denial of
remove "and"
> + errmsg("must be a superuser to log information about specified process")));
I think it should say not say "specified", since that sounds like the user
might have access to log information about some other processes:
| must be a superuser to log information about processes
> +
> + proc = BackendPidGetProc(pid);
> +
> + /*
> + * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
> + * we reach kill(), a process for which we get a valid proc here might
> + * have terminated on its own. There's no way to acquire a lock on an
> + * arbitrary process to prevent that. But since this mechanism is usually
> + * used to below purposes, it might end its own first and the information
used for below purposes,
--
Justin
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-12 18:37 ` Re: RFC: Logging plan of the running query Justin Pryzby <[email protected]>
@ 2021-11-15 14:00 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2021-11-15 14:00 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: [email protected]
On 2021-11-13 03:37, Justin Pryzby wrote:
> I reviewed this version of the patch - I have some language fixes.
Thanks for your review!
Attached patch that reflects your comments.
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v13-0001-log-running-query-plan.patch (27.7K, ../../[email protected]/2-v13-0001-log-running-query-plan.patch)
download | inline diff:
From b8367e22d7a9898e4b85627ba8c203be273fc22f Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 15 Nov 2021 22:31:00 +0900
Subject: [PATCH v13] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby
---
doc/src/sgml/func.sgml | 45 +++++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 117 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 55 +++++++++
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 36 +-----
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 ++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 54 +++++++--
src/test/regress/sql/misc_functions.sql | 42 +++++--
18 files changed, 355 insertions(+), 61 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 24447c0017..7ffaa9a55d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25345,6 +25345,26 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25458,6 +25478,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the plan of the most deeply nested query is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 54c93b16c4..d7f0010e47 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -701,6 +701,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..d930200987 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +44,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1597,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1607,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4922,3 +4928,110 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query",
+ MyProcPid));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is holding a page lock. Try again",
+ MyProcPid));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN));
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 6e69398cdd..6ea63e0c53 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..7aeec69ea5 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,57 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * By default, only superusers are allowed to signal to log information
+ * because allowing any users to issue this request at an unbounded
+ * rate would cause lots of log messages which can lead to denial of
+ * service. Additional roles can be permitted with GRANT.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used for below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ errmsg("PID %d is not a PostgreSQL server process", pid));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ errmsg("could not send signal to process %d: %m", pid));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c25af7fe09..8a70a6d94e 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0775abe35d..774dadb87a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4289,6 +4293,9 @@ PostgresMain(const char *dbname, const char *username)
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..2db662282f 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -175,37 +175,5 @@ Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
-
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
-
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT));
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d068d6532e..0b96cf61b2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4545', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 90a3016065..0fb0da77e2 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 71d316cad3..1b6bc7a89d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,21 +134,22 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -156,15 +157,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -173,8 +174,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 8c23874b3f..5cb2a64aad 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -31,35 +31,55 @@ SELECT num_nonnulls();
SELECT num_nulls();
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
-SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
-CREATE ROLE regress_log_memory;
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log;
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
base-commit: b0f7425ec2445678f32381de8bd3174d3cc2167e
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
@ 2021-11-13 13:29 ` Bharath Rupireddy <[email protected]>
2021-11-15 12:59 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2 siblings, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-11-13 13:29 UTC (permalink / raw)
To: Ekaterina Sokolova <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; torikoshia <[email protected]>
On Wed, Oct 13, 2021 at 7:58 PM Ekaterina Sokolova
<[email protected]> wrote:
> Thank you for working on this issue. I would be glad to continue to
> follow the development of this issue.
Thanks for the patch. I'm not sure if v11 is the latest patch, if yes,
I have the following comments:
1) Firstly, v11 patch isn't getting applied on the master -
http://cfbot.cputube.org/patch_35_3142.log.
2) I think we are moving away from if (!superuser()) checks, see the
commit [1]. The goal is to let the GRANT-REVOKE system deal with who
is supposed to run these system functions. Since
pg_log_current_query_plan also writes the info to server logs, I think
it should do the same thing as commit [1] did for
pg_log_backend_memory_contexts.
With v11, you are re-introducing the superuser() check in the
pg_log_backend_memory_contexts which is wrong.
3) I think SendProcSignalForLogInfo can be more generic, meaning, it
can also send signal to auxiliary processes if asked to do this will
simplify the things for pg_log_backend_memory_contexts and other
patches like pg_print_backtrace. I would imagine it to be "bool
SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason, bool
signal_aux_proc);".
[1] commit f0b051e322d530a340e62f2ae16d99acdbcb3d05
Author: Jeff Davis <[email protected]>
Date: Tue Oct 26 13:13:52 2021 -0700
Allow GRANT on pg_log_backend_memory_contexts().
Remove superuser check, allowing any user granted permissions on
pg_log_backend_memory_contexts() to log the memory contexts of any
backend.
Note that this could allow a privileged non-superuser to log the
memory contexts of a superuser backend, but as discussed, that does
not seem to be a problem.
Reviewed-by: Nathan Bossart, Bharath Rupireddy, Michael Paquier,
Kyotaro Horiguchi, Andres Freund
Discussion:
https://postgr.es/m/[email protected]
Regards,
Bharath Rupireddy.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-13 13:29 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-11-15 12:59 ` torikoshia <[email protected]>
2021-11-15 14:15 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-11-15 12:59 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: Ekaterina Sokolova <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2021-11-13 22:29, Bharath Rupireddy wrote:
Thanks for your review!
> On Wed, Oct 13, 2021 at 7:58 PM Ekaterina Sokolova
> <[email protected]> wrote:
>> Thank you for working on this issue. I would be glad to continue to
>> follow the development of this issue.
>
> Thanks for the patch. I'm not sure if v11 is the latest patch, if yes,
> I have the following comments:
>
> 1) Firstly, v11 patch isn't getting applied on the master -
> http://cfbot.cputube.org/patch_35_3142.log.
Updated the patch.
> 2) I think we are moving away from if (!superuser()) checks, see the
> commit [1]. The goal is to let the GRANT-REVOKE system deal with who
> is supposed to run these system functions. Since
> pg_log_current_query_plan also writes the info to server logs, I think
> it should do the same thing as commit [1] did for
> pg_log_backend_memory_contexts.
>
> With v11, you are re-introducing the superuser() check in the
> pg_log_backend_memory_contexts which is wrong.
Yeah, I removed superuser() check and make it possible to be executed by
non-superusers when users are granted to do so.
>
> 3) I think SendProcSignalForLogInfo can be more generic, meaning, it
> can also send signal to auxiliary processes if asked to do this will
> simplify the things for pg_log_backend_memory_contexts and other
> patches like pg_print_backtrace. I would imagine it to be "bool
> SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason, bool
> signal_aux_proc);".
I agree with your idea.
Since sending signals to auxiliary processes to dump memory contexts and
pg_print_backtrace is still under discussion, IMHO it would be better to
refactor SendProcSignalForLogInfo after these patches are commited.
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v12-0001-log-running-query-plan.patch (27.0K, ../../[email protected]/2-v12-0001-log-running-query-plan.patch)
download | inline diff:
From 5499167a7ecc6f040d5fec817cf36a7ba0b5cbff Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 15 Nov 2021 21:20:43 +0900
Subject: [PATCH v12] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_current_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_current_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda, Ekaterina Sokolova
---
doc/src/sgml/func.sgml | 45 +++++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 117 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 55 +++++++++
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 36 +-----
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 54 +++++++--
src/test/regress/sql/misc_functions.sql | 42 +++++--
17 files changed, 333 insertions(+), 61 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 24447c0017..e12e1feeca 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25345,6 +25345,26 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_current_query_plan</primary>
+ </indexterm>
+ <function>pg_log_current_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25458,6 +25478,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_current_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_current_query_plan(201116);
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the deepest nesting query's plan is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 54c93b16c4..039c15d581 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -701,6 +701,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_current_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..30493b6fa2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +44,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1597,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1607,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4922,3 +4928,110 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is not running a query",
+ MyProcPid)));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("backend with PID %d is now holding a page lock. Try again",
+ MyProcPid)));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("plan of the query running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true)));
+}
+
+/*
+ * pg_log_current_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_current_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN));
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 6e69398cdd..6ea63e0c53 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..97e18ee4cb 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,57 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * By default, only superusers are allowed to signal to log information
+ * because allowing any users to issue this request at an unbounded
+ * rate would cause lots of log messages and which can lead to denial of
+ * service. Additional roles can be permitted with GRANT.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used to below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c25af7fe09..8a70a6d94e 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0775abe35d..774dadb87a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4289,6 +4293,9 @@ PostgresMain(const char *dbname, const char *username)
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..2db662282f 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -175,37 +175,5 @@ Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
-
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
-
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT));
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d068d6532e..5763b45f23 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4545', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_current_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_current_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 90a3016065..0fb0da77e2 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 71d316cad3..918460ee35 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,21 +134,22 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -156,15 +157,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -173,8 +174,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log;
+-- pg_log_current_query_plan()
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_current_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_current_query_plan(integer)
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
+ 'pg_log_current_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log;
+SELECT pg_log_current_query_plan(pg_backend_pid());
+ pg_log_current_query_plan
+---------------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_current_query_plan(integer)
+ FROM regress_log;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 8c23874b3f..462b66a770 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -31,35 +31,55 @@ SELECT num_nonnulls();
SELECT num_nulls();
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
-SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
-CREATE ROLE regress_log_memory;
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log;
+
+-- pg_log_current_query_plan()
+SELECT pg_log_current_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_current_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_current_query_plan(integer)
+ TO regress_log;
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_current_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log;
+SELECT pg_log_current_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_current_query_plan(integer)
+ FROM regress_log;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
base-commit: b0f7425ec2445678f32381de8bd3174d3cc2167e
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-13 13:29 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-11-15 12:59 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-11-15 14:15 ` Bharath Rupireddy <[email protected]>
2021-11-16 11:48 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Bharath Rupireddy @ 2021-11-15 14:15 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Ekaterina Sokolova <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, Nov 15, 2021 at 6:29 PM torikoshia <[email protected]> wrote:
> > 3) I think SendProcSignalForLogInfo can be more generic, meaning, it
> > can also send signal to auxiliary processes if asked to do this will
> > simplify the things for pg_log_backend_memory_contexts and other
> > patches like pg_print_backtrace. I would imagine it to be "bool
> > SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason, bool
> > signal_aux_proc);".
>
> I agree with your idea.
> Since sending signals to auxiliary processes to dump memory contexts and
> pg_print_backtrace is still under discussion, IMHO it would be better to
> refactor SendProcSignalForLogInfo after these patches are commited.
+1.
I have another comment: isn't it a good idea that an overloaded
version of the new function pg_log_query_plan can take the available
explain command options as a text argument? I'm not sure if it is
possible to get the stats like buffers, costs etc of a running query,
if yes, something like pg_log_query_plan(pid, 'buffers',
'costs'....);? It looks to be an overkill at first sight, but these
can be useful to know more detailed plan of the query.
Thoughts?
Regards,
Bharath Rupireddy.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-13 13:29 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-11-15 12:59 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-15 14:15 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-11-16 11:48 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2021-11-16 11:48 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: Ekaterina Sokolova <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2021-11-15 23:15, Bharath Rupireddy wrote:
> I have another comment: isn't it a good idea that an overloaded
> version of the new function pg_log_query_plan can take the available
> explain command options as a text argument? I'm not sure if it is
> possible to get the stats like buffers, costs etc of a running query,
> if yes, something like pg_log_query_plan(pid, 'buffers',
> 'costs'....);? It looks to be an overkill at first sight, but these
> can be useful to know more detailed plan of the query.
I also think the overloaded version would be useful.
However as discussed in [1], it seems to introduce other difficulties.
I think it would be enough that the first version of pg_log_query_plan
doesn't take any parameters.
[1]
https://www.postgresql.org/message-id/ce86e4f72f09d5497e8ad3a162861d33%40oss.nttdata.com
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
@ 2021-05-13 10:12 ` torikoshia <[email protected]>
1 sibling, 0 replies; 124+ messages in thread
From: torikoshia @ 2021-05-13 10:12 UTC (permalink / raw)
To: Bharath Rupireddy <[email protected]>; +Cc: [email protected]; pgsql-hackers
On 2021-05-13 18:36, Bharath Rupireddy wrote:
> On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy
> <[email protected]> wrote:
>> On Thu, May 13, 2021 at 2:44 PM Dilip Kumar <[email protected]>
>> wrote:
>> > +1 for the idea. I did not read the complete patch but while reading
>> > through the patch, I noticed that you using elevel as LOG for printing
>> > the stack trace. But I think the backend whose pid you have passed,
>> > the connected client to that backend might not have superuser
>> > privileges and if you use elevel LOG then that message will be sent to
>> > that connected client as well and I don't think that is secure. So
>> > can we use LOG_SERVER_ONLY so that we can prevent
>> > it from sending to the client.
>>
>> True, we should use LOG_SERVER_ONLY and not send any logs to the
>> client.
Thanks, agree with changing it to LOG_SERVER_ONLY.
> I further tend to think that, is it correct to log queries with LOG
> level when log_statement GUC is set? Or should it also be
> LOG_SERVER_ONLY?
I feel it's OK to log with LOG_SERVER_ONLY since the log from
log_statement GUC would be printed already and independently.
ISTM people don't expect to log_statement GUC works even on
pg_log_current_plan(), do they?
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
@ 2021-11-02 11:32 Ekaterina Sokolova <[email protected]>
2021-11-03 12:48 ` Re: RFC: Logging plan of the running query Daniel Gustafsson <[email protected]>
2021-11-04 12:49 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
0 siblings, 3 replies; 124+ messages in thread
From: Ekaterina Sokolova @ 2021-11-02 11:32 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]
Hi!
I'm here to answer your questions about contrib/pg_query_state.
>> I only took a quick look at pg_query_state, I have some questions.
>> pg_query_state seems using shm_mq to expose the plan information, but
>> there was a discussion that this kind of architecture would be tricky
>> to do properly [1].
>> Does pg_query_state handle difficulties listed on the discussion?
> [1]
> https://www.postgresql.org/message-id/9a50371e15e741e295accabc72a41df1%40oss.nttdata.com
I doubt that it was the right link.
But on the topic I will say that extension really use shared memory,
interaction is implemented by sending / receiving messages. This
architecture provides the required reliability and convenience.
>> It seems the caller of the pg_query_state() has to wait until the
>> target process pushes the plan information into shared memory, can it
>> lead to deadlock situations?
>> I came up with this question because when trying to make a view for
>> memory contexts of other backends, we encountered deadlock situations.
>> After all, we gave up view design and adopted sending signal and
>> logging.
>
> Discussion at the following URL.
> https://www.postgresql.org/message-id/9a50371e15e741e295accabc72a41df1%40oss.nttdata.com
Before extracting information about side process we check its state.
Information will only be retrieved for a process willing to provide it.
Otherwise, we will receive an error message about impossibility of
getting query execution statistics + process status. Also checking fact
of extracting your own status exists. This is even verified in tests.
Thanks for your attention.
Just in case, I am ready to discuss this topic in more detail.
About overhead:
> I haven't measured it yet, but I believe that the overhead for backends
> which are not called pg_log_current_plan() would be slight since the
> patch just adds the logic for saving QueryDesc on ExecutorRun().
> The overhead for backends which is called pg_log_current_plan() might
> not slight, but since the target process are assumed dealing with
> long-running query and the user want to know its plan, its overhead
> would be worth the cost.
I think it would be useful for us to have couple of examples with a
different number of rows compared to using without this functionality.
Hope this helps.
--
Ekaterina Sokolova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
@ 2021-11-03 12:48 ` Daniel Gustafsson <[email protected]>
2 siblings, 0 replies; 124+ messages in thread
From: Daniel Gustafsson @ 2021-11-03 12:48 UTC (permalink / raw)
To: Ekaterina Sokolova <[email protected]>; +Cc: torikoshia <[email protected]>; [email protected]
This patch no longer applies on top of HEAD, please submit a rebased version.
--
Daniel Gustafsson https://vmware.com/
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
@ 2021-11-04 12:49 ` torikoshia <[email protected]>
2021-11-17 13:44 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2021-11-04 12:49 UTC (permalink / raw)
To: Ekaterina Sokolova <[email protected]>; +Cc: [email protected]
On 2021-11-02 20:32, Ekaterina Sokolova wrote:
Thanks for your response!
> Hi!
>
> I'm here to answer your questions about contrib/pg_query_state.
>>> I only took a quick look at pg_query_state, I have some questions.
>
>>> pg_query_state seems using shm_mq to expose the plan information, but
>>> there was a discussion that this kind of architecture would be tricky
>>> to do properly [1].
>>> Does pg_query_state handle difficulties listed on the discussion?
>> [1]
>> https://www.postgresql.org/message-id/9a50371e15e741e295accabc72a41df1%40oss.nttdata.com
>
> I doubt that it was the right link.
Sorry for make you confused, here is the link.
https://www.postgresql.org/message-id/CA%2BTgmobkpFV0UB67kzXuD36--OFHwz1bs%3DL_6PZbD4nxKqUQMw%40mail...
> But on the topic I will say that extension really use shared memory,
> interaction is implemented by sending / receiving messages. This
> architecture provides the required reliability and convenience.
As described in the link, using shared memory for this kind of work
would need DSM and It'd be also necessary to exchange information
between requestor and responder.
For example, when I looked at a little bit of pg_query_state code, it
looks like the size of the queue is fixed at QUEUE_SIZE, and I wonder
how plans that exceed QUEUE_SIZE are handled.
>>> It seems the caller of the pg_query_state() has to wait until the
>>> target process pushes the plan information into shared memory, can it
>>> lead to deadlock situations?
>>> I came up with this question because when trying to make a view for
>>> memory contexts of other backends, we encountered deadlock
>>> situations.
>>> After all, we gave up view design and adopted sending signal and
>>> logging.
>>
>> Discussion at the following URL.
>> https://www.postgresql.org/message-id/9a50371e15e741e295accabc72a41df1%40oss.nttdata.com
>
> Before extracting information about side process we check its state.
> Information will only be retrieved for a process willing to provide
> it. Otherwise, we will receive an error message about impossibility of
> getting query execution statistics + process status. Also checking
> fact of extracting your own status exists. This is even verified in
> tests.
>
> Thanks for your attention.
> Just in case, I am ready to discuss this topic in more detail.
I imagined the following procedure.
Does it cause dead lock in pg_query_state?
- session1
BEGIN; TRUNCATE t;
- session2
BEGIN; TRUNCATE t; -- wait
- session1
SELECT * FROM pg_query_state(<pid of session>); -- wait and dead locked?
> About overhead:
>> I haven't measured it yet, but I believe that the overhead for
>> backends
>> which are not called pg_log_current_plan() would be slight since the
>> patch just adds the logic for saving QueryDesc on ExecutorRun().
>> The overhead for backends which is called pg_log_current_plan() might
>> not slight, but since the target process are assumed dealing with
>> long-running query and the user want to know its plan, its overhead
>> would be worth the cost.
> I think it would be useful for us to have couple of examples with a
> different number of rows compared to using without this functionality.
Do you have any expectaion that the number of rows would affect the
performance of this functionality?
This patch adds some codes to ExecutorRun(), but I thought the number of
rows would not give impact on the performance.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-04 12:49 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2021-11-17 13:44 ` Ekaterina Sokolova <[email protected]>
2021-11-26 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Ekaterina Sokolova @ 2021-11-17 13:44 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]
Hi!
You forgot my last fix to build correctly on Mac. I have added it.
About our discussion of pg_query_state:
torikoshia писал 2021-11-04 15:49:
>> I doubt that it was the right link.
> Sorry for make you confused, here is the link.
> https://www.postgresql.org/message-id/CA%2BTgmobkpFV0UB67kzXuD36--OFHwz1bs%3DL_6PZbD4nxKqUQMw%40mail...
Thank you. I'll see it soon.
> I imagined the following procedure.
> Does it cause dead lock in pg_query_state?
>
> - session1
> BEGIN; TRUNCATE t;
>
> - session2
> BEGIN; TRUNCATE t; -- wait
>
> - session1
> SELECT * FROM pg_query_state(<pid of session>); -- wait and dead
> locked?
As I know, pg_query_state use non-blocking read and write. I have wrote
few tests trying to deadlock it (on 14 version), but all finished
correctly.
Have a nice day. Please feel free to contact me if you need any further
information.
--
Ekaterina Sokolova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
[text/x-diff] v13-0002-log-running-query-plan.patch (28.4K, ../../[email protected]/2-v13-0002-log-running-query-plan.patch)
download | inline diff:
From b8367e22d7a9898e4b85627ba8c203be273fc22f Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 15 Nov 2021 22:31:00 +0900
Subject: [PATCH v13] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby
---
doc/src/sgml/func.sgml | 45 +++++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 117 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 55 +++++++++
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 36 +-----
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 ++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 54 +++++++--
src/test/regress/sql/misc_functions.sql | 42 +++++--
19 files changed, 355 insertions(+), 63 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 24447c0017..7ffaa9a55d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25345,6 +25345,26 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25458,6 +25478,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the plan of the most deeply nested query is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 54c93b16c4..d7f0010e47 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -701,6 +701,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 10644dfac4..d930200987 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +44,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1597,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1607,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4922,3 +4928,110 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query",
+ MyProcPid));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is holding a page lock. Try again",
+ MyProcPid));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN));
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 6e69398cdd..6ea63e0c53 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..7aeec69ea5 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,57 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * By default, only superusers are allowed to signal to log information
+ * because allowing any users to issue this request at an unbounded
+ * rate would cause lots of log messages which can lead to denial of
+ * service. Additional roles can be permitted with GRANT.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used for below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ errmsg("PID %d is not a PostgreSQL server process", pid));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ errmsg("could not send signal to process %d: %m", pid));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c25af7fe09..8a70a6d94e 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0775abe35d..774dadb87a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4289,6 +4293,9 @@ PostgresMain(const char *dbname, const char *username)
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..2db662282f 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -175,37 +175,5 @@ Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
-
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
-
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT));
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d068d6532e..0b96cf61b2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8038,6 +8038,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4545', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 90a3016065..0fb0da77e2 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index a5286fab893..4f0eaf83fbc 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 71d316cad3..1b6bc7a89d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,21 +134,22 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -156,15 +157,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -173,8 +174,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 8c23874b3f..5cb2a64aad 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -31,35 +31,55 @@ SELECT num_nonnulls();
SELECT num_nulls();
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
-SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
-CREATE ROLE regress_log_memory;
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log;
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
base-commit: b0f7425ec2445678f32381de8bd3174d3cc2167e
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-04 12:49 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-17 13:44 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
@ 2021-11-26 03:39 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2021-11-26 03:39 UTC (permalink / raw)
To: Ekaterina Sokolova <[email protected]>; +Cc: [email protected]
On 2021-11-17 22:44, Ekaterina Sokolova wrote:
> Hi!
>
> You forgot my last fix to build correctly on Mac. I have added it.
Thanks for the notification!
Since the patch could not be applied to the HEAD anymore, I also updated
it.
>
> About our discussion of pg_query_state:
>
> torikoshia писал 2021-11-04 15:49:
>>> I doubt that it was the right link.
>> Sorry for make you confused, here is the link.
>> https://www.postgresql.org/message-id/CA%2BTgmobkpFV0UB67kzXuD36--OFHwz1bs%3DL_6PZbD4nxKqUQMw%40mail...
>
> Thank you. I'll see it soon.
>
>> I imagined the following procedure.
>> Does it cause dead lock in pg_query_state?
>>
>> - session1
>> BEGIN; TRUNCATE t;
>>
>> - session2
>> BEGIN; TRUNCATE t; -- wait
>>
>> - session1
>> SELECT * FROM pg_query_state(<pid of session>); -- wait and dead
>> locked?
>
> As I know, pg_query_state use non-blocking read and write. I have
> wrote few tests trying to deadlock it (on 14 version), but all
> finished correctly.
>
> Have a nice day. Please feel free to contact me if you need any
> further information.
Thanks for your information and help!
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v14-0001-log-running-query-plan.patch (28.4K, ../../[email protected]/2-v14-0001-log-running-query-plan.patch)
download | inline diff:
From b8367e22d7a9898e4b85627ba8c203be273fc22f Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Fri, 26 Nov 2021 10:31:00 +0900
Subject: [PATCH v14] Add function to log the untruncated query string and its
plan for the query currently running on the backend with the specified
process ID.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Since some codes, tests and comments of
pg_log_query_plan() are the same with
pg_log_backend_memory_contexts(), this patch also refactors
them to make them common.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar, Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby
---
doc/src/sgml/func.sgml | 45 +++++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 117 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/ipc/signalfuncs.c | 55 +++++++++
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 7 ++
src/backend/utils/adt/mcxtfuncs.c | 36 +-----
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/storage/signalfuncs.h | 22 ++++
src/include/tcop/pquery.h | 1 +
src/test/regress/expected/misc_functions.out | 54 +++++++--
src/test/regress/sql/misc_functions.sql | 42 +++++--
19 files changed, 355 insertions(+), 63 deletions(-)
create mode 100644 src/include/storage/signalfuncs.h
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0a725a6711..b84ead4341 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25358,6 +25358,26 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID along with the untruncated
+ query string.
+ They will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25471,6 +25491,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: plan of the query running on backend with PID 17793 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that nested statements (statements executed inside a function) are not
+ considered for logging. Only the plan of the most deeply nested query is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index f6789025a5..a2288b60c9 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -707,6 +707,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 09f5253abb..41a6d89b80 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +44,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1597,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed, as the target query may use instrumentation and clean
+ * itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1607,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4925,3 +4931,110 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogCurrentPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogCurrentPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogCurrentPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogCurrentPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogCurrentPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogCurrentPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogCurrentPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogCurrentPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query",
+ MyProcPid));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is holding a page lock. Try again",
+ MyProcPid));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->running = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_CURRENT_PLAN));
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b3ce4bae53..c74aa1d04e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 6e69398cdd..6ea63e0c53 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_CURRENT_PLAN))
+ HandleLogCurrentPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..7aeec69ea5 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,57 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * Signal a backend process to log its information.
+ *
+ * By default, only superusers are allowed to signal to log information
+ * because allowing any users to issue this request at an unbounded
+ * rate would cause lots of log messages which can lead to denial of
+ * service. Additional roles can be permitted with GRANT.
+ *
+ * On receipt of this signal, a backend sets the flag in the signal
+ * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
+ * information.
+ */
+bool
+SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason)
+{
+ PGPROC *proc;
+
+ Assert(reason == PROCSIG_LOG_MEMORY_CONTEXT ||
+ reason == PROCSIG_LOG_CURRENT_PLAN);
+
+ proc = BackendPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might
+ * have terminated on its own. There's no way to acquire a lock on an
+ * arbitrary process to prevent that. But since this mechanism is usually
+ * used for below purposes, it might end its own first and the information
+ * is not logged is not a problem.
+ * - debug a backend running and consuming lots of memory
+ * - look into the plan of the long running query
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ errmsg("PID %d is not a PostgreSQL server process", pid));
+ return false;
+ }
+
+ if (SendProcSignal(pid, reason, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ errmsg("could not send signal to process %d: %m", pid));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c25af7fe09..8a70a6d94e 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0775abe35d..774dadb87a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "executor/spi.h"
#include "jit/jit.h"
@@ -3366,6 +3367,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogCurrentPlanPending)
+ ProcessLogCurrentPlanInterrupt();
}
@@ -4289,6 +4293,9 @@ PostgresMain(const char *dbname, const char *username)
/* We don't have a transaction command open anymore */
xact_started = false;
+ /* We have no running query */
+ ActiveQueryDesc = NULL;
+
/*
* If an error occurred while we were reading a message from the
* client, we have potentially lost track of where the previous
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..2db662282f 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -18,8 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
-#include "storage/proc.h"
#include "storage/procarray.h"
+#include "storage/signalfuncs.h"
#include "utils/builtins.h"
/* ----------
@@ -175,37 +175,5 @@ Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
int pid = PG_GETARG_INT32(0);
- PGPROC *proc;
-
- proc = BackendPidGetProc(pid);
-
- /*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might
- * have terminated on its own. There's no way to acquire a lock on an
- * arbitrary process to prevent that. But since this mechanism is usually
- * used to debug a backend running and consuming lots of memory, that it
- * might end on its own first and its memory contexts are not logged is
- * not a problem.
- */
- if (proc == NULL)
- {
- /*
- * This is just a warning so a loop-through-resultset will not abort
- * if one backend terminated on its own during the run.
- */
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
- }
-
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
- {
- /* Again, just a warning to allow loops */
- ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
- PG_RETURN_BOOL(false);
- }
-
- PG_RETURN_BOOL(true);
+ PG_RETURN_BOOL(SendProcSignalForLogInfo(pid, PROCSIG_LOG_MEMORY_CONTEXT));
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..126ed10362 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogCurrentPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e934361dc3..2fed1ea5e2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8042,6 +8042,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4545', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7c2dce7cdf 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is already running */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogCurrentPlanInterrupt(void);
+extern void ProcessLogCurrentPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 90a3016065..0fb0da77e2 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogCurrentPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index a5286fab89..4f0eaf83fb 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..c1a7218d69 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_CURRENT_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/storage/signalfuncs.h b/src/include/storage/signalfuncs.h
new file mode 100644
index 0000000000..5f6b124372
--- /dev/null
+++ b/src/include/storage/signalfuncs.h
@@ -0,0 +1,22 @@
+/*-------------------------------------------------------------------------
+ *
+ * signalfuncs.h
+ * Functions for signaling backends
+ *
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/signalfuncs.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCSIGNALFUNC_H
+#define PROCSIGNALFUNC_H
+
+/*
+ * prototypes for functions in signalfuncs.c
+ */
+extern bool SendProcSignalForLogInfo(pid_t pid, ProcSignalReason reason);
+
+#endif /* PROCSIGNALFUNC_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 2318f04ff0..d37a5eb837 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 1013d17f87..876052034d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -134,21 +134,22 @@ LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -156,15 +157,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -173,8 +174,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log;
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 7ab9b2a150..b4dd725072 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -31,35 +31,55 @@ SELECT num_nonnulls();
SELECT num_nulls();
--
--- pg_log_backend_memory_contexts()
+-- Test logging functions
--
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
---
-SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+-- pg_log_backend_memory_contexts()
+CREATE ROLE regress_log;
-CREATE ROLE regress_log_memory;
+SELECT pg_log_backend_memory_contexts(pg_backend_pid());
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log;
+
+SELECT has_function_privilege('regress_log',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log;
--
-- Test some built-in SRFs
base-commit: 99e4d24a9d77e7bb87e15b318e96dc36651a7da2
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
@ 2025-03-08 15:42 ` Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2 siblings, 1 reply; 124+ messages in thread
From: Akshat Jaimini @ 2025-03-08 15:42 UTC (permalink / raw)
To: [email protected]; +Cc: Atsushi Torikoshi <[email protected]>
Hi,
I think there is still some problem with the patch. I am not able to apply it to the master branch.
Can you please take another look at it?
Thanks,
Akshat Jaimini
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
@ 2025-03-10 05:10 ` torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-03-10 05:10 UTC (permalink / raw)
To: Akshat Jaimini <[email protected]>; +Cc: [email protected]
On 2025-03-09 00:42, Akshat Jaimini wrote:
> Hi,
> I think there is still some problem with the patch. I am not able to
> apply it to the master branch.
>
> Can you please take another look at it?
Thanks for pointing it out!
Modified it.
BTW the patch adds about 400 lines to explain.c and it may be better to
split the file as well as 9173e8b6046, but I leave it as it is for now.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v41-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (32.9K, ../../[email protected]/2-v41-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 699264fbc1f4e99114966eaeba55a0ec5184e1c2 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 10 Mar 2025 14:01:54 +0900
Subject: [PATCH v41] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, codes for logging plan is wrapped
to every ExecProcNode and when the executor runs one of
ExecProcNode, the plan is actually logged. These wrappers are
unwrapped when once the plan is logged.
In this way, we can avoid adding the overhead which we'll face
when adding CHECK_FOR_INTERRUPTS() like mechanisms in somewhere
in executor codes safely.
---
contrib/auto_explain/auto_explain.c | 23 +-
doc/src/sgml/func.sgml | 50 +++
src/backend/access/transam/xact.c | 7 +
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 372 ++++++++++++++++++-
src/backend/executor/execMain.c | 12 +
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 12 +
src/include/miscadmin.h | 1 +
src/include/nodes/execnodes.h | 3 +
src/include/storage/procsignal.h | 2 +
src/include/tcop/pquery.h | 2 +-
src/test/regress/expected/misc_functions.out | 54 ++-
src/test/regress/sql/misc_functions.sql | 41 +-
17 files changed, 555 insertions(+), 42 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 7007a226c0..85cfe1b4f4 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -408,26 +408,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainAssembleLogOutput(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 51dd8ad657..d1a16a89fb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28553,6 +28553,25 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -28671,6 +28690,37 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+ Query Identifier: 8621255546560739680
+</screen>
+ Note that when the target is executing nested statements(statements executed
+ inside a function), only the innermost query plan is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 1b4f21a88d..ca973a0368 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -2897,6 +2898,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset pg_log_query_plan() related state. */
+ ResetLogQueryPlanState();
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5285,6 +5289,9 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /* Reset pg_log_query_plan() related state. */
+ ResetLogQueryPlanState();
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 566f308e44..ec2e1c1fd9 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -775,6 +775,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index d8a7232ced..7f63f47fce 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -24,6 +24,7 @@
#include "jit/jit.h"
#include "libpq/pqformat.h"
#include "libpq/protocol.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -31,7 +32,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
#include "tcop/tcopprot.h"
+#include "utils/backend_status.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
#include "utils/json.h"
@@ -43,6 +46,11 @@
#include "utils/typcache.h"
#include "utils/xml.h"
+bool ProcessLogQueryPlanInterruptActive = false;
+
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -154,6 +162,9 @@ static ExplainWorkersState *ExplainCreateWorkersState(int num_workers);
static void ExplainOpenWorker(int n, ExplainState *es);
static void ExplainCloseWorker(int n, ExplainState *es);
static void ExplainFlushWorkersState(ExplainState *es);
+static TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+static void WrapExecProcNodeWithExplain(PlanState *ps);
+static void UnwrapExecProcNodeWithExplain(PlanState *ps);
@@ -875,6 +886,37 @@ ExplainPrintSettings(ExplainState *es)
}
}
+/*
+ * ExplainAssembleLogOutput -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainAssembleLogOutput(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* ExplainPrintPlan -
* convert a QueryDesc's plan tree to text and append it to es->str
@@ -1947,6 +1989,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been executed
+ * and explain has been called by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1954,7 +1999,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4931,3 +4976,328 @@ ExplainFlushWorkersState(ExplainState *es)
pfree(wstate->worker_state_save);
pfree(wstate);
}
+
+/*
+ * HandleLogQueryPlanInterrupt -
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ResetLogQueryPlanState -
+ * Clear pg_log_query_plan() related state during (sub)transaction abort
+ */
+void
+ResetLogQueryPlanState(void)
+{
+ /*
+ * After abort, some elements of ActiveQueryDesc is freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * WrapMultiExecProcNodesWithExplain -
+ * Wrap array of PlanStates ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapMultiExecProcNodesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ WrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * WrapCustomPlanChildExecProcNodesWithExplain -
+ * Wrap CustomScanstate children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapCustomPlanChildExecProcNodesWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ WrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * WrapExecProcNodeWithExplain -
+ * Wrap ExecProcNode with ExecProcNodeWithExplain recursively
+ */
+static void
+WrapExecProcNodeWithExplain(PlanState *ps)
+{
+ /* wrapping can be done only once */
+ if (ps->ExecProcNodeOriginal != NULL)
+ return;
+
+ check_stack_depth();
+
+ ps->ExecProcNodeOriginal = ps->ExecProcNode;
+ ps->ExecProcNode = ExecProcNodeWithExplain;
+
+ if (ps->lefttree != NULL)
+ WrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ WrapExecProcNodeWithExplain(ps->righttree);
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ereport(DEBUG1, errmsg("wrapping Append"));
+ WrapMultiExecProcNodesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ereport(DEBUG1, errmsg("wrapping MergeAppend"));
+ WrapMultiExecProcNodesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ereport(DEBUG1, errmsg("wrapping BitmapAndState"));
+ WrapMultiExecProcNodesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ereport(DEBUG1, errmsg("wrapping BitmapOrtate"));
+ WrapMultiExecProcNodesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ereport(DEBUG1, errmsg("wrapping Subquery"));
+ WrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ ereport(DEBUG1, errmsg("wrapping CustomScanState"));
+ WrapCustomPlanChildExecProcNodesWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * UnwrapMultiExecProcNodesWithExplain -
+ * Unwrap array of PlanStates ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapMultiExecProcNodesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ UnwrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * UnwrapCustomPlanChildExecProcNodesWithExplain -
+ * Unwrap CustomScanstate children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapCustomPlanChildExecProcNodesWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ UnwrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * UnwrapExecProcNodeWithExplain -
+ * Unwrap ExecProcNode with ExecProcNodeWithExplain recursively
+ */
+static void
+UnwrapExecProcNodeWithExplain(PlanState *ps)
+{
+ Assert(ps->ExecProcNodeOriginal != NULL);
+
+ check_stack_depth();
+
+ ps->ExecProcNode = ps->ExecProcNodeOriginal;
+ ps->ExecProcNodeOriginal = NULL;
+
+ if (ps->lefttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->righttree);
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ereport(DEBUG1, errmsg("unwrapping Append"));
+ UnwrapMultiExecProcNodesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ereport(DEBUG1, errmsg("unwrapping MergeAppend"));
+ UnwrapMultiExecProcNodesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ereport(DEBUG1, errmsg("unwrapping BitmapAndState"));
+ UnwrapMultiExecProcNodesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ereport(DEBUG1, errmsg("unwrapping BitmapOrtate"));
+ UnwrapMultiExecProcNodesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ereport(DEBUG1, errmsg("unwrapping Subquery"));
+ UnwrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ ereport(DEBUG1, errmsg("unwrapping CustomScanState"));
+ UnwrapCustomPlanChildExecProcNodesWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * ExecProcNodeWithExplain -
+ * Wrap ExecProcNode with codes which logs currently running plan
+ */
+static TupleTableSlot *
+ExecProcNodeWithExplain(PlanState *ps)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+
+ check_stack_depth();
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainAssembleLogOutput(es, ActiveQueryDesc, EXPLAIN_FORMAT_TEXT, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ UnwrapExecProcNodeWithExplain(ActiveQueryDesc->planstate);
+
+ /*
+ * Since unwrapping has already done, call ExecProcNode() not
+ * ExecProcNodeOriginal().
+ */
+ return ps->ExecProcNode(ps);
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Add wrapper which logs explain of the plan to ExecProcNodes
+ *
+ * Since running EXPLAIN codes at any arbitrary CHECK_FOR_INTERRUPTS() is
+ * unsafe, this function just wraps every ExecProcNode.
+ * In this way, EXPLAIN code is only executed at the timing of ExecProcNode,
+ * which seems safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ LogQueryPlanPending = false;
+
+ /* Cannot re-enter */
+ if (ProcessLogQueryPlanInterruptActive)
+ return;
+
+ ProcessLogQueryPlanInterruptActive = true;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+
+ WrapExecProcNodeWithExplain(ActiveQueryDesc->planstate);
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 0493b7d536..63284517fa 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -362,10 +363,21 @@ void
ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count)
{
+ /*
+ * Update ActiveQueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count);
else
standard_ExecutorRun(queryDesc, direction, count);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 7d20196550..ced80588ab 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -690,6 +691,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 947ffb4042..f4c4e64f9f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3499,6 +3500,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index b844f9fdae..1e4d90f58b 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -39,6 +39,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cede992b6e..c8ed884109 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8499,6 +8499,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 64547bd9b9..47c8376a60 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -17,6 +17,9 @@
#include "lib/stringinfo.h"
#include "parser/parse_node.h"
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
+extern PGDLLIMPORT bool ProcessLogQueryPlanInterruptActive;
+
typedef enum ExplainSerializeOption
{
EXPLAIN_SERIALIZE_NONE,
@@ -71,6 +74,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -112,6 +116,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, CachedPlan *cplan,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainAssembleLogOutput(ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
+
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
@@ -120,4 +128,8 @@ extern void ExplainPrintJITSummary(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern void ResetLogQueryPlanState(void);
+
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index a2b63495ee..291c20187d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a323fa98bb..73431a3223 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1160,6 +1160,9 @@ typedef struct PlanState
ExecProcNodeMtd ExecProcNodeReal; /* actual function, if above is a
* wrapper */
+ ExecProcNodeMtd ExecProcNodeOriginal; /* temporary place when adding
+ * process for ExecProcNode */
+
Instrumentation *instrument; /* Optional runtime stats for this node */
WorkerInstrumentation *worker_instrument; /* per-worker instrumentation */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 022fd8ed93..c666a4d708 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index fa3cc5f2df..73e16cf7c4 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,7 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index d3f5d16a67..7dcb9951e7 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -316,14 +316,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -337,8 +338,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -346,15 +347,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -363,8 +364,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index aaebb29833..92330355f6 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -109,39 +109,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: e033696596566d422a0eae47adca371a210ed730
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-03-21 12:40 ` torikoshia <[email protected]>
2025-03-31 18:51 ` Re: RFC: Logging plan of the running query Sadeq Dousti <[email protected]>
2025-03-31 19:24 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-04-03 05:59 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
0 siblings, 4 replies; 124+ messages in thread
From: torikoshia @ 2025-03-21 12:40 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
Hi,
Rebased it again.
On 2025-03-10 14:10, torikoshia wrote:
> BTW the patch adds about 400 lines to explain.c and it may be better
> to split the file as well as 9173e8b6046, but I leave it as it is for
> now.
This part remains unchanged.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v42-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (33.4K, ../../[email protected]/2-v42-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From ea5092a13e7ac426ac40397ceff5f67355b7723f Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Fri, 21 Mar 2025 21:34:08 +0900
Subject: [PATCH v42] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, codes for logging plan is wrapped
to every ExecProcNode and when the executor runs one of
ExecProcNode, the plan is actually logged. These wrappers are
unwrapped when once the plan is logged.
In this way, we can avoid adding the overhead which we'll face
when adding CHECK_FOR_INTERRUPTS() like mechanisms in somewhere
in executor codes safely.
---
contrib/auto_explain/auto_explain.c | 23 +-
doc/src/sgml/func.sgml | 50 +++
src/backend/access/transam/xact.c | 7 +
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 372 ++++++++++++++++++-
src/backend/executor/execMain.c | 12 +
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 13 +
src/include/commands/explain_state.h | 1 +
src/include/miscadmin.h | 1 +
src/include/nodes/execnodes.h | 3 +
src/include/storage/procsignal.h | 2 +
src/include/tcop/pquery.h | 2 +-
src/test/regress/expected/misc_functions.out | 54 ++-
src/test/regress/sql/misc_functions.sql | 41 +-
18 files changed, 557 insertions(+), 42 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 3b73bd1910..5807135000 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -409,26 +409,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainAssembleLogOutput(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6fa1d6586b..da80b7ef8a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28570,6 +28570,25 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -28688,6 +28707,37 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+ Query Identifier: 8621255546560739680
+</screen>
+ Note that when the target is executing nested statements(statements executed
+ inside a function), only the innermost query plan is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b885513f76..1b4c0abea5 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -2904,6 +2905,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset pg_log_query_plan() related state. */
+ ResetLogQueryPlanState();
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5296,6 +5300,9 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /* Reset pg_log_query_plan() related state. */
+ ResetLogQueryPlanState();
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 566f308e44..ec2e1c1fd9 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -775,6 +775,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 391b34a2af..bdfe545e76 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -26,6 +26,7 @@
#include "jit/jit.h"
#include "libpq/pqformat.h"
#include "libpq/protocol.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -33,7 +34,9 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
#include "tcop/tcopprot.h"
+#include "utils/backend_status.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
#include "utils/json.h"
@@ -45,6 +48,11 @@
#include "utils/typcache.h"
#include "utils/xml.h"
+bool ProcessLogQueryPlanInterruptActive = false;
+
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -165,6 +173,9 @@ static ExplainWorkersState *ExplainCreateWorkersState(int num_workers);
static void ExplainOpenWorker(int n, ExplainState *es);
static void ExplainCloseWorker(int n, ExplainState *es);
static void ExplainFlushWorkersState(ExplainState *es);
+static TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+static void WrapExecProcNodeWithExplain(PlanState *ps);
+static void UnwrapExecProcNodeWithExplain(PlanState *ps);
@@ -756,6 +767,37 @@ ExplainPrintSettings(ExplainState *es)
}
}
+/*
+ * ExplainAssembleLogOutput -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainAssembleLogOutput(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* ExplainPrintPlan -
* convert a QueryDesc's plan tree to text and append it to es->str
@@ -1828,6 +1870,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been executed
+ * and explain has been called by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1880,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4987,3 +5032,328 @@ ExplainFlushWorkersState(ExplainState *es)
pfree(wstate->worker_state_save);
pfree(wstate);
}
+
+/*
+ * HandleLogQueryPlanInterrupt -
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ResetLogQueryPlanState -
+ * Clear pg_log_query_plan() related state during (sub)transaction abort
+ */
+void
+ResetLogQueryPlanState(void)
+{
+ /*
+ * After abort, some elements of ActiveQueryDesc is freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * WrapMultiExecProcNodesWithExplain -
+ * Wrap array of PlanStates ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapMultiExecProcNodesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ WrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * WrapCustomPlanChildExecProcNodesWithExplain -
+ * Wrap CustomScanstate children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapCustomPlanChildExecProcNodesWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ WrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * WrapExecProcNodeWithExplain -
+ * Wrap ExecProcNode with ExecProcNodeWithExplain recursively
+ */
+static void
+WrapExecProcNodeWithExplain(PlanState *ps)
+{
+ /* wrapping can be done only once */
+ if (ps->ExecProcNodeOriginal != NULL)
+ return;
+
+ check_stack_depth();
+
+ ps->ExecProcNodeOriginal = ps->ExecProcNode;
+ ps->ExecProcNode = ExecProcNodeWithExplain;
+
+ if (ps->lefttree != NULL)
+ WrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ WrapExecProcNodeWithExplain(ps->righttree);
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ereport(DEBUG1, errmsg("wrapping Append"));
+ WrapMultiExecProcNodesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ereport(DEBUG1, errmsg("wrapping MergeAppend"));
+ WrapMultiExecProcNodesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ereport(DEBUG1, errmsg("wrapping BitmapAndState"));
+ WrapMultiExecProcNodesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ereport(DEBUG1, errmsg("wrapping BitmapOrtate"));
+ WrapMultiExecProcNodesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ereport(DEBUG1, errmsg("wrapping Subquery"));
+ WrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ ereport(DEBUG1, errmsg("wrapping CustomScanState"));
+ WrapCustomPlanChildExecProcNodesWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * UnwrapMultiExecProcNodesWithExplain -
+ * Unwrap array of PlanStates ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapMultiExecProcNodesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ UnwrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * UnwrapCustomPlanChildExecProcNodesWithExplain -
+ * Unwrap CustomScanstate children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapCustomPlanChildExecProcNodesWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ UnwrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * UnwrapExecProcNodeWithExplain -
+ * Unwrap ExecProcNode with ExecProcNodeWithExplain recursively
+ */
+static void
+UnwrapExecProcNodeWithExplain(PlanState *ps)
+{
+ Assert(ps->ExecProcNodeOriginal != NULL);
+
+ check_stack_depth();
+
+ ps->ExecProcNode = ps->ExecProcNodeOriginal;
+ ps->ExecProcNodeOriginal = NULL;
+
+ if (ps->lefttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->righttree);
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ereport(DEBUG1, errmsg("unwrapping Append"));
+ UnwrapMultiExecProcNodesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ereport(DEBUG1, errmsg("unwrapping MergeAppend"));
+ UnwrapMultiExecProcNodesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ereport(DEBUG1, errmsg("unwrapping BitmapAndState"));
+ UnwrapMultiExecProcNodesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ereport(DEBUG1, errmsg("unwrapping BitmapOrtate"));
+ UnwrapMultiExecProcNodesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ereport(DEBUG1, errmsg("unwrapping Subquery"));
+ UnwrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ ereport(DEBUG1, errmsg("unwrapping CustomScanState"));
+ UnwrapCustomPlanChildExecProcNodesWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * ExecProcNodeWithExplain -
+ * Wrap ExecProcNode with codes which logs currently running plan
+ */
+static TupleTableSlot *
+ExecProcNodeWithExplain(PlanState *ps)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+
+ check_stack_depth();
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainAssembleLogOutput(es, ActiveQueryDesc, EXPLAIN_FORMAT_TEXT, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ UnwrapExecProcNodeWithExplain(ActiveQueryDesc->planstate);
+
+ /*
+ * Since unwrapping has already done, call ExecProcNode() not
+ * ExecProcNodeOriginal().
+ */
+ return ps->ExecProcNode(ps);
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Add wrapper which logs explain of the plan to ExecProcNodes
+ *
+ * Since running EXPLAIN codes at any arbitrary CHECK_FOR_INTERRUPTS() is
+ * unsafe, this function just wraps every ExecProcNode.
+ * In this way, EXPLAIN code is only executed at the timing of ExecProcNode,
+ * which seems safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ LogQueryPlanPending = false;
+
+ /* Cannot re-enter */
+ if (ProcessLogQueryPlanInterruptActive)
+ return;
+
+ ProcessLogQueryPlanInterruptActive = true;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+
+ WrapExecProcNodeWithExplain(ActiveQueryDesc->planstate);
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index e9bd98c773..755387d70b 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -362,10 +363,21 @@ void
ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count)
{
+ /*
+ * Update ActiveQueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count);
else
standard_ExecutorRun(queryDesc, direction, count);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 7d20196550..ced80588ab 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -690,6 +691,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0554a4ae3c..cf97f2030b 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3507,6 +3508,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index b844f9fdae..1e4d90f58b 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -39,6 +39,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 890822eaf7..f47359fe3c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8509,6 +8509,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 387839eb5d..dba686ac97 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -18,6 +18,9 @@
struct ExplainState; /* defined in explain_state.h */
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
+extern PGDLLIMPORT bool ProcessLogQueryPlanInterruptActive;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -72,6 +75,12 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, CachedPlan *cplan,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainAssembleLogOutput(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
+
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
@@ -83,4 +92,8 @@ extern void ExplainQueryText(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(struct ExplainState *es,
ParamListInfo params, int maxlen);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern void ResetLogQueryPlanState(void);
+
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 32728f5d1a..fcef956396 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 603d042435..3f46354cda 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index d4d4e65518..b95e4b97d0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1160,6 +1160,9 @@ typedef struct PlanState
ExecProcNodeMtd ExecProcNodeReal; /* actual function, if above is a
* wrapper */
+ ExecProcNodeMtd ExecProcNodeOriginal; /* temporary place when adding
+ * process for ExecProcNode */
+
Instrumentation *instrument; /* Optional runtime stats for this node */
WorkerInstrumentation *worker_instrument; /* per-worker instrumentation */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 022fd8ed93..c666a4d708 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index fa3cc5f2df..73e16cf7c4 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,7 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index d3f5d16a67..7dcb9951e7 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -316,14 +316,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -337,8 +338,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -346,15 +347,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -363,8 +364,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index aaebb29833..92330355f6 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -109,39 +109,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: 1d617a20284f887cb9cdfe5693eec155e8016517
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-03-31 18:51 ` Sadeq Dousti <[email protected]>
2025-04-03 05:34 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
3 siblings, 1 reply; 124+ messages in thread
From: Sadeq Dousti @ 2025-03-31 18:51 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]; Sergey Dudoladov <[email protected]>
Hi,
Sergey and I reviewed this patch and have a few comments, listed below.
> BTW the patch adds about 400 lines to explain.c and it may be better
> to split the file as well as 9173e8b6046, but I leave it as it is for
> now.
1. As rightfully described by the OP above, explain.c has grown too big. In
addition, explain.h is added to quite a number of other files.
2. We wanted to entertain the possibility of a GUC variable to enable the
feature. That is, having it disabled by default, and the DBA can
selectively enable it on the fly. The reasoning is that it can affect some
workloads in an unforeseen way, so this choice can make the next Postgres
release safer. In future releases, we can make the default "enabled",
assuming enough real-world scenarios have proven the feature safe (i.e.,
not affecting the DB performance noticeably).
3. In the email thread for this patch, we saw some attempts to measure the
performance overhead of the feature. We suggest a more rigorous study,
including memory overhead in addition to time overhead. We came to the
conclusion that analytical workloads - with complicated query plans - and a
high query rate are the best to attempt such benchmarks.
4. In PGConf EU 2024, there was a talk by Rafael Castro titled "Debugging
active queries" [1], and he also submitted a recent patch titled "Proposal:
Progressive explain" [2]. We see a lot of similarities in the idea behind
that patch and this one, and would like to suggest joining forces for an
ideal outcome.
[1] https://www.youtube.com/watch?v=6ahTb-7C05c
[2] https://commitfest.postgresql.org/patch/5473/
Best Regards,
Sadeq Dousti & Sergey Dudoladov
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-31 18:51 ` Re: RFC: Logging plan of the running query Sadeq Dousti <[email protected]>
@ 2025-04-03 05:34 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2025-04-03 05:34 UTC (permalink / raw)
To: Sadeq Dousti <[email protected]>; +Cc: [email protected]; Sergey Dudoladov <[email protected]>
On 2025-04-01 03:51, Sadeq Dousti wrote:
> Hi,
>
> Sergey and I reviewed this patch and have a few comments, listed
> below.
Thanks for your review!
>> BTW the patch adds about 400 lines to explain.c and it may be better
>> to split the file as well as 9173e8b6046, but I leave it as it is
> for
>> now.
>
> 1. As rightfully described by the OP above, explain.c has grown too
> big. In addition, explain.h is added to quite a number of other files.
Agreed.
>
> 2. We wanted to entertain the possibility of a GUC variable to enable
> the feature. That is, having it disabled by default, and the DBA can
> selectively enable it on the fly. The reasoning is that it can affect
> some workloads in an unforeseen way, so this choice can make the next
> Postgres release safer. In future releases, we can make the default
> "enabled", assuming enough real-world scenarios have proven the
> feature safe (i.e., not affecting the DB performance noticeably).
Agreed.
> 3. In the email thread for this patch, we saw some attempts to measure
> the performance overhead of the feature. We suggest a more rigorous
> study, including memory overhead in addition to time overhead. We came
> to the conclusion that analytical workloads - with complicated query
> plans - and a high query rate are the best to attempt such benchmarks.
Agreed.
> 4. In PGConf EU 2024, there was a talk by Rafael Castro titled
> "Debugging active queries" [1], and he also submitted a recent patch
> titled "Proposal: Progressive explain" [2]. We see a lot of
> similarities in the idea behind that patch and this one, and would
> like to suggest joining forces for an ideal outcome.
As we have been discussing in this thread recently, I believe that since
this patch has a narrower scope, it makes sense to complete it first and
then extend it to support progress tracking in a step-by-step manner.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-03-31 19:24 ` Robert Haas <[email protected]>
3 siblings, 0 replies; 124+ messages in thread
From: Robert Haas @ 2025-03-31 19:24 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]; [email protected]; Rafael Thofehrn Castro <[email protected]>
On Fri, Mar 21, 2025 at 8:40 AM torikoshia <[email protected]> wrote:
> Rebased it again.
Hi,
I apologize for not having noticed this thread sooner. I just became
aware of it as a result of a discussion in the hacking Discord server.
I think this has got a lot over overlap with the progressive EXPLAIN
patch from Rafael Castro, which I have been reviewing, but I am not
sure why we have two different patch sets here. Why is that?
Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-04-03 05:59 ` torikoshia <[email protected]>
3 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2025-04-03 05:59 UTC (permalink / raw)
To: Sami Imseih <[email protected]>; +Cc: Robert Haas <[email protected]>; [email protected]; [email protected]
On 2025-04-03 01:22, Sami Imseih wrote:
>> > FWIW, I have been thinking about auto_explain for another task,
>> > remote plans for fdw [0], and perhaps there are now other good
>> > reasons, some that you mention, that can be simplified if "auto_explain"
>> > becomes a core feature. This could be a proposal taken up in 19.
>>
>> For what we're talking about here, I don't think we would need to go
>> that far -- maybe put a few functions in core but no real need to move
>> the whole module into core. However, I don't rule out that there are
>> other reasons to do as you suggest.
>
> This is the first core feature that will allow users to log explain
> plans for
> a live workload. EXPLAIN is not really good for this purpose. So this
> proposal is a step in the correct direction. I think the appetite for
> more
> plan logging/visibility options will likely increase, and
> auto_explainlike
> features in core will be desired IMO. We will see.
>
> As far as this patch goes, I took a look and I have some comments:
Thanks for your comments!
> 2/
> It should be noted that the plan will not print to the log until
> the plan begins executing the next plan node? depending on the
> operation, that could take some time ( i.e.long seq scan of a table,
> etc.)
> Does this behavior need to be called out in docs?
Seems reasonable, but long seq scan of a table would not cause the case
since ExecProcNode() is called at least the number of the rows in a
table, as far as I remember.
Of course there can be the case where long time can elapse before
executing ExecProcNode(), so I agree with adding the doc about this.
I'm going to improve including other than this comment in the next
patch.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-04-05 06:13 ` Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
3 siblings, 1 reply; 124+ messages in thread
From: Atsushi Torikoshi @ 2025-04-05 06:13 UTC (permalink / raw)
To: Robert Haas <[email protected]>; [email protected]; +Cc: torikoshia <[email protected]>; [email protected]; [email protected]
On Thu, Apr 3, 2025 at 11:10 PM Robert Haas <[email protected]> wrote:
> Looking at ExplainAssembleLogOutput() is making me realize that
> auto_explain is in serious need of some cleanup. That's not really the
> fault of this patch, but the hack whereby we overwrite the [] that
> would have surrounded the JSON output with {} is not very nice. I also
> think that the auto_explain GUCs need rethinking. In theory, new
> EXPLAIN options should be mirrored into auto_explain, but if you
> compare ExplainAssembleLogOutput() to ExplainOnePlan(), you can see
> that they are diverging. The PLANNING, SUMMARY, and SERIALIZE options
> that are known to regular EXPLAIN aren't known to auto_explain, and
> any customizable options that use explain_per_plan_hook won't be able
> to work with auto_explain, either. Changing this is non-trivial
> because SERIALIZE, for example, can't work the same way for
> auto_explain as it does for EXPLAIN, and a full solution might also
> require user-visible changes like replacing
> auto_explain.log_<whatever> with auto_explain.explain, so I don't
> really know. Maybe we should just live with it the way it is for now,
> but it doesn't look very nice.
I agree that the current state isn’t ideal, and that addressing it wouldn’t
be trivial.
While I do think it would be better to tackle this before proceeding with
the current patch, I don’t think it's a prerequisite.
Also, since I’m not yet sure what the best way to fix it would be, I’ve
left it as-is for now.
> I don't think it's a good idea to put the logic to update
> ActiveQueryDesc into ExecutorRun. I think that function should really
> just call the hook, or standard_ExecutorRun. I don't know for sure
> whether this work should be moved down into ExecutorRun or up into the
> caller, but I don't think it should stay where it is.
Moved the logic from ExecutorRun() to standard_ExecutorRun(), since it
performs the necessary setup for collecting information during plan
execution i.e. calling InstrStartNode().
> My comment about the naming of WrapMultiExecProcNodesWithExplain() on
> the other thread also applies here: MultiExecProcNode() is an
> unrelated function.
Renamed WrapMultiExecProcNodesWithExplain to WrapPlanStatesWithExplain.
> Likewise, WrapExecProcNodeWithExplain() misses
> walking the node's subplan list.
Added logic for subplan to WrapExecProcNodeWithExplain() and
UnwrapExecProcNodeWithExplain().
> Also, I don't think an
> ereport(DEBUG1, ...) is appropriate here.
Removed these ereport(DEBUG1).
> Do we really need es->signaled? Couldn't we test es->analyze instead?
I think it's necessary.
Although when implementing progressive EXPLAIN, I believe we can use
es->analyze instead, this patch aims to retrieve plans for queries that
have neither an EXPLAIN nor an ANALYZE specified.
> Do we really need ExecProcNodeOriginal? Can we find some way to reuse
> ExecProcNodeReal instead of making the structure bigger?
I also wanted to implement this without adding elements to PlanState if
possible, but I haven't found a good solution, so the patch uses
ExecSetExecProcNode.
> I do think we should try to move some of this new code out into a
> separate source file, but I'm not yet sure what we should call it. We
> might want to share infrastructure with something what Rafael's patch,
> which he called progressive EXPLAIN but which is really closer to a
> general query progress facility, albeit perhaps too expensive to have
> enabled by default.
Made new files dynamic_explain.h and dynamic_explain.c, which you named and
it seems fine to me, and move most of the code to them.
> Does the documentation typically mention when a function is
> superuser-only? If so, it should do so here, too, using similar
> wording.
Added below explanation like other functions:
This function is restricted to superusers by default, but other
users can be granted EXECUTE to run the function.
> It seems unnecessary to remark that you can't log a query plan after a
> subtransaction abort, because the query wouldn't be running any more.
Removed the description.
> It's also true that you can't log a query after a toplevel abort, even
> if you're still waiting for the aborted transaction to roll back. But
> it also seems like once the aborted subtransaction is popped off the
> stack and we return to the parent transaction, we should be able to
> log any query running at the outer level.
It works as below:
session-1
begin;
savepoint s1;
err;
ERROR: syntax error at or near "err"
rollback to s1;
select pg_sleep(5);
session-2
select pg_log_query_plan(pid of session-1);
log
LOG: query plan running on backend with PID 40025 is:
Query Text: select pg_sleep(5);
Result (cost=0.00..0.01 rows=1 width=4)
Output: pg_sleep('5'::double precision)
> Does ActiveQueryDesc really need to be exposed to the whole system?
> Could it be a file-level variable?
On Thu, Apr 3, 2025 at 11:10 PM Robert Haas <[email protected]> wrote:
> I think the question of whether ActiveQueryDesc can be file-level is
> separate from whether prior configuration is needed. If it is
> important to touch this from multiple source files, then it is fine
> for it to be global. However, if we have a new source file, say
> dynamic_explain.c, then you could have functions
> ProcessDynamicExplainInterrupt() and DynamicExplainCleanup() in that
> file to set, use, clear ActiveQueryDesc, and the rest of the system
> might not need to know about it.
Thanks for the advice, I took this approach and made ActiveQueryDesc
file-level variable.
On Thu, Apr 3, 2025 at 1:23 AM Sami Imseih <[email protected]> wrote:
> 7/
> Why do we only support TEXT format? I tried it with JSON
> and it looks fine in the log as well. I can imagine automated
> tools will want to be able to retrieve the plans using the
> structured formats.
I agree that allowing the choice of format would be useful.
However, I believe implementing this would require introducing new GUC or
creating a mechanism to pass the format from the executor of
pg_log_query_plan() to the target process.
For now, I think it would be better to minimize the scope of the initial
patch.
I think I have reflected your other comments in the attached patch.
Regards,
Atsushi Torikoshi
Attachments:
[application/octet-stream] v43-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (35.5K, ../../CAM6-o=BecZ-FOOHZXLhMU=JGpGQBPS5cA=ASoLUr92fFgjvQqg@mail.gmail.com/3-v43-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 183046805796954b775f6d02b310760dec293523 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Sat, 5 Apr 2025 15:00:40 +0900
Subject: [PATCH v43] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, codes for logging plan is wrapped
to every ExecProcNode and when the executor runs one of
ExecProcNode, the plan is actually logged. These wrappers are
unwrapped when once the plan is logged.
In this way, we can avoid adding the overhead which we'll face
when adding CHECK_FOR_INTERRUPTS() like mechanisms in somewhere
in executor codes safely.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
---
contrib/auto_explain/auto_explain.c | 24 +-
doc/src/sgml/func.sgml | 55 +++
src/backend/access/transam/xact.c | 7 +
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 368 +++++++++++++++++++
src/backend/commands/explain.c | 38 +-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 10 +
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 26 ++
src/include/commands/explain.h | 5 +
src/include/commands/explain_state.h | 1 +
src/include/miscadmin.h | 1 +
src/include/nodes/execnodes.h | 3 +
src/include/storage/procsignal.h | 2 +
src/include/tcop/pquery.h | 1 -
src/test/regress/expected/misc_functions.out | 54 ++-
src/test/regress/sql/misc_functions.sql | 41 ++-
22 files changed, 613 insertions(+), 43 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index cd6625020a..e720ddf39f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -412,26 +413,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9f531e2328..a71689fe5d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28666,6 +28666,29 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -28784,6 +28807,38 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+ Query Identifier: 8621255546560739680
+</screen>
+ Note that when the target is executing nested statements(statements executed
+ inside a function), only the innermost query plan is logged.
+ Note that logging plan may take some time, as it occurs when
+ the plan node is executed. For example, when a query is
+ running <function>pg_sleep</function>, the plan will not be
+ logged until the function execution completes.
+ Similarly, when a query is running under the extended query
+ protocol, the plan is logged only during the execute step.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b885513f76..b49edd2366 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -2904,6 +2905,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset pg_log_query_plan() related state. */
+ ResetLogQueryPlanState();
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5296,6 +5300,9 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /* Reset pg_log_query_plan() related state. */
+ ResetLogQueryPlanState();
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 566f308e44..ec2e1c1fd9 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -775,6 +775,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c6..5e83907eb1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 0000000000..d161aa8fa7
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,368 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+
+/* Whether this backend is performing logging plan */
+static bool ProcessLogQueryPlanInterruptActive = false;
+
+/* Currently executing query's QueryDesc */
+static QueryDesc *ActiveQueryDesc = NULL;
+
+static TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+static void WrapExecProcNodeWithExplain(PlanState *ps);
+static void UnwrapExecProcNodeWithExplain(PlanState *ps);
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Clear pg_log_query_plan() related state during (sub)transaction abort
+ */
+void
+ResetLogQueryPlanState(void)
+{
+ /*
+ * After abort, some elements of ActiveQueryDesc is freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * Wrap array of PlanState ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapPlanStatesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ WrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapCustomPlanChildWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ WrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * Wrap ExecProcNode with ExecProcNodeWithExplain recursively
+ */
+static void
+WrapExecProcNodeWithExplain(PlanState *ps)
+{
+ /* wrapping can be done only once */
+ if (ps->ExecProcNodeOriginal != NULL)
+ return;
+
+ check_stack_depth();
+
+ ps->ExecProcNodeOriginal = ps->ExecProcNode;
+ ps->ExecProcNode = ExecProcNodeWithExplain;
+
+ if (ps->lefttree != NULL)
+ WrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ WrapExecProcNodeWithExplain(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ WrapExecProcNodeWithExplain(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ WrapPlanStatesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ WrapPlanStatesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ WrapPlanStatesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ WrapPlanStatesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ WrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ WrapCustomPlanChildWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Unwrap array of PlanStates ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapPlanStatesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ UnwrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * Unwrap CustomScanState children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapCustomPlanChildWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ UnwrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * Unwrap ExecProcNode with ExecProcNodeWithExplain recursively
+ */
+static void
+UnwrapExecProcNodeWithExplain(PlanState *ps)
+{
+ Assert(ps->ExecProcNodeOriginal != NULL);
+
+ check_stack_depth();
+
+ ps->ExecProcNode = ps->ExecProcNodeOriginal;
+ ps->ExecProcNodeOriginal = NULL;
+
+ if (ps->lefttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ UnwrapExecProcNodeWithExplain(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ UnwrapPlanStatesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ UnwrapPlanStatesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ UnwrapPlanStatesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ UnwrapPlanStatesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ UnwrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ UnwrapCustomPlanChildWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Wrap ExecProcNode with codes which logs currently running plan
+ */
+static TupleTableSlot *
+ExecProcNodeWithExplain(PlanState *ps)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+
+ check_stack_depth();
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainStringAssemble(es, ActiveQueryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ UnwrapExecProcNodeWithExplain(ActiveQueryDesc->planstate);
+
+ /*
+ * Since unwrapping has already done, call ExecProcNode() not
+ * ExecProcNodeOriginal().
+ */
+ return ps->ExecProcNode(ps);
+}
+
+/*
+ * Add wrapper which logs explain of the plan to ExecProcNode
+ *
+ * Since running EXPLAIN codes at any arbitrary CHECK_FOR_INTERRUPTS() is
+ * unsafe, this function just wraps every ExecProcNode.
+ * In this way, EXPLAIN code is only executed at the timing of ExecProcNode,
+ * which seems safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ LogQueryPlanPending = false;
+
+ /* Cannot re-enter */
+ if (ProcessLogQueryPlanInterruptActive)
+ return;
+
+ ProcessLogQueryPlanInterruptActive = true;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+
+ WrapExecProcNodeWithExplain(ActiveQueryDesc->planstate);
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+QueryDesc *
+GetActiveQueryDesc(void)
+{
+ return ActiveQueryDesc;
+}
+
+void
+SetActiveQueryDesc(QueryDesc *queryDesc)
+{
+ ActiveQueryDesc = queryDesc;
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index ef8aa489af..c0bce0df49 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1100,6 +1100,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1827,7 +1858,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been called by signal, as the target query may use instrumentation
+ * and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1869,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d3..4a64b8e9d0 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 2da848970b..455c0a1c14 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -380,6 +381,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldActiveQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -393,6 +395,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save ActiveQueryDesc here to enable retrieval of the currently running
+ * queryDesc for nested queries.
+ */
+ oldActiveQueryDesc = GetActiveQueryDesc();
+ SetActiveQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -455,6 +464,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetActiveQueryDesc(oldActiveQueryDesc);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index b7c39a4c5f..af8dc5a9fb 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -690,6 +691,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8918984886..5c430237f2 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/dynamic_explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3531,6 +3532,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d..583d621ad6 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a28a15993a..237650c99e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8535,6 +8535,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 0000000000..68b16ec9be
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,26 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ResetLogQueryPlanState(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern QueryDesc *GetActiveQueryDesc(void);
+extern void SetActiveQueryDesc(QueryDesc *queryDesc);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 387839eb5d..da7f580be7 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -72,6 +72,8 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, CachedPlan *cplan,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
@@ -82,5 +84,8 @@ extern void ExplainPrintJITSummary(struct ExplainState *es,
extern void ExplainQueryText(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(struct ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 32728f5d1a..fcef956396 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0d8528b287..d3cc8e7e75 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 5b6cadb5a6..feb29a3818 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1166,6 +1166,9 @@ typedef struct PlanState
ExecProcNodeMtd ExecProcNodeReal; /* actual function, if above is a
* wrapper */
+ ExecProcNodeMtd ExecProcNodeOriginal; /* temporary place when adding
+ * process for ExecProcNode */
+
Instrumentation *instrument; /* Optional runtime stats for this node */
WorkerInstrumentation *worker_instrument; /* per-worker instrumentation */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 016dfd9b3f..56e1ff3949 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index fa3cc5f2df..7949abf76b 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -22,7 +22,6 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
extern PortalStrategy ChoosePortalStrategy(List *stmts);
extern List *FetchPortalTargetList(Portal portal);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index d3f5d16a67..7dcb9951e7 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -316,14 +316,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -337,8 +338,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -346,15 +347,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -363,8 +364,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index aaebb29833..92330355f6 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -109,39 +109,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: 67be093562b6b345c170417312dff22f467055ba
--
2.43.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
@ 2025-04-27 23:55 ` Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Hannu Krosing @ 2025-04-27 23:55 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; Robert Haas <[email protected]>; [email protected]; [email protected]; [email protected]
Have you also checked out
https://github.com/postgrespro/pg_query_state which logs running query
plan AND collected counts and timings as a response to a signal?
Has this ever been discussed for inclusion in core ?
On Thu, Apr 24, 2025 at 2:49 PM torikoshia <[email protected]> wrote:
>
> Hi,
>
> Attached a rebased version of the patch.
>
> --
> Regards,
>
> --
> Atsushi Torikoshi
> Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
@ 2025-05-20 13:17 ` torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-05-20 13:17 UTC (permalink / raw)
To: [email protected]; [email protected]; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Sat, Apr 5, 2025 at 3:14 PM Atsushi Torikoshi
<[email protected]> wrote:
> On Thu, Apr 3, 2025 at 11:10 PM Robert Haas <[email protected]>
> wrote:
>> Do we really need ExecProcNodeOriginal? Can we find some way to reuse
>> ExecProcNodeReal instead of making the structure bigger?
> I also wanted to implement this without adding elements to PlanState if
> possible, but I haven't found a good solution, so the patch uses
> ExecSetExecProcNode.
I tackled this again and the attached patch removes ExecProcNodeOriginal
from Planstate.
Instead of adding a new field, this version builds the behavior into the
existing wrapper function, ExecProcNodeFirst().
Since ExecProcNodeFirst() is already handling instrumentation-related
logic, the patch has maybe become a bit more complex to accommodate both
that and the new behavior.
While it might make sense to introduce a more general mechanism that
allows for stacking an arbitrary number of wrappers around ExecProcNode,
I’m not sure it's possible or worth the added complexity—such layered
wrapping doesn't seem like something we typically need.
What do you think?
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v45-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.6K, ../../[email protected]/2-v45-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 41944eb943f8f6b2fb731125ed0d50ad29bbd338 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 20 May 2025 22:01:40 +0900
Subject: [PATCH v45] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, codes for logging plan is wrapped
to every ExecProcNode under the current executing plan node,
and when the executor runs one of ExecProcNode, the plan is
actually logged. These wrappers are unwrapped when once the
plan is logged.
In this way, we can avoid adding the overhead which we'll face
when adding CHECK_FOR_INTERRUPTS() like mechanisms in somewhere
in executor codes safely.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
---
contrib/auto_explain/auto_explain.c | 24 +-
doc/src/sgml/func.sgml | 55 +++
src/backend/access/transam/xact.c | 13 +
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 387 +++++++++++++++++++
src/backend/commands/explain.c | 38 +-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 10 +
src/backend/executor/execProcnode.c | 13 +-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 28 ++
src/include/commands/explain.h | 5 +
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/include/tcop/pquery.h | 1 -
src/test/regress/expected/misc_functions.out | 54 ++-
src/test/regress/sql/misc_functions.sql | 41 +-
23 files changed, 648 insertions(+), 46 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index cd6625020a..e720ddf39f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -412,26 +413,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b405525a46..700fdde184 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28821,6 +28821,29 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -28971,6 +28994,38 @@ stats_timestamp | 2025-03-24 13:55:47.796698+01
will be less resource intensive when only the local backend is of interest.
</para>
</note>
+ </para>
+
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+ Query Identifier: 8621255546560739680
+</screen>
+ Note that when the target is executing nested statements(statements executed
+ inside a function), only the innermost query plan is logged.
+ Note that logging plan may take some time, as it occurs when
+ the plan node is executed. For example, when a query is
+ running <function>pg_sleep</function>, the plan will not be
+ logged until the function execution completes.
+ Similarly, when a query is running under the extended query
+ protocol, the plan is logged only during the execute step.
</para>
</sect2>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b885513f76..6ba9b8f824 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -2904,6 +2905,12 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /*
+ * After abort, some elements of ActiveQueryDesc are freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ResetLogQueryPlanState();
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5296,6 +5303,12 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * After abort, some elements of ActiveQueryDesc are freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ResetLogQueryPlanState();
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 566f308e44..ec2e1c1fd9 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -775,6 +775,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c6..5e83907eb1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 0000000000..55629ea554
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,387 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+
+/*
+ * True while this backend is processing a log query plan request,
+ * from the start of wrapping plan nodes until the log output is completed.
+ */
+static bool ProcessLogQueryPlanInterruptActive = false;
+
+/* Currently executing query's QueryDesc */
+static QueryDesc *ActiveQueryDesc = NULL;
+
+static void WrapExecProcNodeWithExplain(PlanState *ps);
+static void UnwrapExecProcNodeWithExplain(PlanState *ps);
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Clear pg_log_query_plan() related state during (sub)transaction abort
+ */
+void
+ResetLogQueryPlanState(void)
+{
+ ActiveQueryDesc = NULL;
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * Wrap array of PlanState ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapPlanStatesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ WrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+WrapCustomPlanChildWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ WrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all possible ExecProcNode().
+ *
+ * Recursion is necessary because the next ExecProcNode() call may be invoked
+ * not only through the current node, but also via lefttree, righttree, subPlan,
+ * or other special child plans.
+ */
+static void
+WrapExecProcNodeWithExplain(PlanState *ps)
+{
+ check_stack_depth();
+
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ WrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ WrapExecProcNodeWithExplain(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+ WrapExecProcNodeWithExplain(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ WrapPlanStatesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ WrapPlanStatesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ WrapPlanStatesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ WrapPlanStatesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ WrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ WrapCustomPlanChildWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Unwrap array of PlanStates ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapPlanStatesWithExplain(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ UnwrapExecProcNodeWithExplain(planstates[i]);
+}
+
+/*
+ * Unwrap CustomScanState children's ExecProcNodes with ExecProcNodeWithExplain
+ */
+static void
+UnwrapCustomPlanChildWithExplain(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ UnwrapExecProcNodeWithExplain((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively unwrap all possible ExecProcNode().
+ *
+ * Unwrap ExecProcNode() or wrap it for instrumentation if needed.
+ * Since ExecProcNodeWithExplain() is wrapped ealier in ExecProcNodeFirst(),
+ * perform instrumentation wrapping in this function.
+ */
+static void
+UnwrapExecProcNodeWithExplain(PlanState *ps)
+{
+ check_stack_depth();
+
+ if (ps->instrument && INSTR_TIME_IS_ZERO(ps->instrument->starttime))
+ ps->ExecProcNode = ExecProcNodeInstr;
+ else
+ ps->ExecProcNode = ps->ExecProcNodeReal;
+
+ if (ps->lefttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->lefttree);
+ if (ps->righttree != NULL)
+ UnwrapExecProcNodeWithExplain(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ UnwrapExecProcNodeWithExplain(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ UnwrapPlanStatesWithExplain(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ UnwrapPlanStatesWithExplain(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ UnwrapPlanStatesWithExplain(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ UnwrapPlanStatesWithExplain(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ UnwrapExecProcNodeWithExplain(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ UnwrapCustomPlanChildWithExplain((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Wrapper for logging currently running plan.
+ *
+ * ExecProcNode wrapper that performs logging plan of the currently running query.
+ */
+TupleTableSlot *
+ExecProcNodeWithExplain(PlanState *ps)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+
+ check_stack_depth();
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * ActiveQueryDesc is valid only during standard_ExecutorRun(). However,
+ * ExecProcNode() can still be called afterward, such as ExecPostprocessPlan().
+ * To handle the case, check ActiveQueryDesc.
+ */
+ if (ActiveQueryDesc == NULL)
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is finishing query",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ else
+ {
+ ExplainStringAssemble(es, ActiveQueryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+ }
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ UnwrapExecProcNodeWithExplain(ps);
+
+ ProcessLogQueryPlanInterruptActive = false;
+
+ return ps->ExecProcNode(ps);
+}
+
+/*
+ * Perform logging plan for the currently running query.
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the ExecProcNode() to
+ * log the query plan. This ensures that EXPLAIN code is executed only during
+ * ExecProcNode(), where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ LogQueryPlanPending = false;
+
+ /* Prevent re-entrance until the plan has been logged and the unwrapping has done */
+ if (ProcessLogQueryPlanInterruptActive)
+ return;
+
+ ProcessLogQueryPlanInterruptActive = true;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+
+ WrapExecProcNodeWithExplain(ActiveQueryDesc->planstate);
+}
+
+bool
+GetProcessLogQueryPlanInterruptActive(void)
+{
+ return ProcessLogQueryPlanInterruptActive;
+}
+
+inline QueryDesc *
+GetActiveQueryDesc(void)
+{
+ return ActiveQueryDesc;
+}
+
+void
+inline SetActiveQueryDesc(QueryDesc *queryDesc)
+{
+ ActiveQueryDesc = queryDesc;
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 786ee865f1..fd37772504 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1100,6 +1100,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1827,7 +1858,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been called by signal, as the target query may use instrumentation
+ * and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1869,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d3..4a64b8e9d0 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 7230f96810..b1f3e83b62 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -380,6 +381,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldActiveQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -393,6 +395,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save ActiveQueryDesc here to enable retrieval of the currently running
+ * queryDesc for nested queries.
+ */
+ oldActiveQueryDesc = GetActiveQueryDesc();
+ SetActiveQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -455,6 +464,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetActiveQueryDesc(oldActiveQueryDesc);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeea..dbdd99830b 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -120,7 +121,6 @@
#include "nodes/nodeFuncs.h"
static TupleTableSlot *ExecProcNodeFirst(PlanState *node);
-static TupleTableSlot *ExecProcNodeInstr(PlanState *node);
static bool ExecShutdownNode_walker(PlanState *node, void *context);
@@ -456,12 +456,19 @@ ExecProcNodeFirst(PlanState *node)
*/
check_stack_depth();
+ /*
+ * If logging plan is requested, handle it first. If instrumentation is also
+ * requested, update the wrapper accordingly after logging plan is completed.
+ * See ExecProcNodeWithExplain().
+ */
+ if (GetProcessLogQueryPlanInterruptActive())
+ node->ExecProcNode = ExecProcNodeWithExplain;
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -475,7 +482,7 @@ ExecProcNodeFirst(PlanState *node)
* this a separate function, we avoid overhead in the normal case where
* no instrumentation is wanted.
*/
-static TupleTableSlot *
+TupleTableSlot *
ExecProcNodeInstr(PlanState *node)
{
TupleTableSlot *result;
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index ce69e26d72..765b5c6599 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -694,6 +695,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_GET_MEMORY_CONTEXT))
HandleGetMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 1ae51b1b39..082a714629 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/dynamic_explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3538,6 +3539,9 @@ ProcessInterrupts(void)
if (PublishMemoryContextPending)
ProcessGetMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 92b0446b80..ffd5f712de 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -41,6 +41,8 @@ volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t PublishMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 62beb71da2..ae793299e2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8581,6 +8581,12 @@
proargnames => '{pid, summary, timeout, name, ident, type, path, level, total_bytes, total_nblocks, free_bytes, free_chunks, used_bytes, num_agg_contexts, stats_timestamp}',
prosrc => 'pg_get_process_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 0000000000..75b864f63a
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,28 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ResetLogQueryPlanState(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern bool GetProcessLogQueryPlanInterruptActive(void);
+extern QueryDesc *GetActiveQueryDesc(void);
+extern void SetActiveQueryDesc(QueryDesc *queryDesc);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 03c5b3d73e..083567caf0 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -72,6 +72,8 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, CachedPlan *cplan,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
@@ -82,5 +84,8 @@ extern void ExplainPrintJITSummary(struct ExplainState *es,
extern void ExplainQueryText(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(struct ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 32728f5d1a..fcef956396 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index ae99407db8..93faf9c811 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -295,6 +295,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
*/
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
+extern TupleTableSlot *ExecProcNodeInstr(PlanState *node);
extern Node *MultiExecProcNode(PlanState *node);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1e59a7f910..2b7d664812 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -97,6 +97,7 @@ extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t PublishMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 345d5a0ecb..60d6d80c4f 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -36,6 +36,8 @@ typedef enum
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
PROCSIG_GET_MEMORY_CONTEXT, /* ask backend to send the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index fa3cc5f2df..7949abf76b 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -22,7 +22,6 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
extern PortalStrategy ChoosePortalStrategy(List *stmts);
extern List *FetchPortalTargetList(Portal portal);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index cc517ed5e9..2c4d2c3178 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -316,14 +316,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -337,8 +338,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -346,15 +347,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -363,8 +364,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 5f9c77512d..d9df928389 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -109,39 +109,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: cbf53e2b8a8ed3fc6f554095a4e99591bd5193f6
--
2.43.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-05-22 15:07 ` Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2025-05-22 15:07 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Tue, May 20, 2025 at 9:18 AM torikoshia <[email protected]> wrote:
> I tackled this again and the attached patch removes ExecProcNodeOriginal
> from Planstate.
> Instead of adding a new field, this version builds the behavior into the
> existing wrapper function, ExecProcNodeFirst().
>
> Since ExecProcNodeFirst() is already handling instrumentation-related
> logic, the patch has maybe become a bit more complex to accommodate both
> that and the new behavior.
>
> While it might make sense to introduce a more general mechanism that
> allows for stacking an arbitrary number of wrappers around ExecProcNode,
> I’m not sure it's possible or worth the added complexity—such layered
> wrapping doesn't seem like something we typically need.
>
> What do you think?
Hmm, I'm not convinced that this is correct. If
GetProcessLogQueryPlanInterruptActive() is true but node->instrument
is also non-NULL, your implementation of ExecProcNodeFirst() will
handle the first but ignore the second. Plus, I don't understand why
ExecProcNodeFirst() does node->ExecProcNode = ExecProcNodeWithExplain
instead of just directly doing the necessary work. It seems to me that
this will result in the first call to ExecProcNode calling
ExecProcNodeFirst to install ExecProcNodeWithExplain; and then the
second call to ExecProcNode will call ExecProcNodeWithExplain which
will actually do the work. But this seems unnecessary to me: I think
it could just say if (GetProcessLogQueryPlanInterruptActive())
LogQueryPlan(ps).
Backing up a step, I think you've got a good idea here in thinking
that we can probably reuse ExecProcNodeFirst for this purpose. It
appears to me that it is always valid to reset a node's ExecProcNode
pointer back to ExecProcNodeFirst. It might be unnecessary and cost
some performance to do it when not required, but it is safe, because
ExecProcNodeFirst will simply reset node->ExecProcNode and then call
the appropriate function. So what we can do is just add the additional
logic to check whether we need to print the query plan after the
check_stack_depth() call and before the rest of the logic in the
function. I've attached a sample patch to show what I have in mind.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] example-execprocnodefirst-patch.diff (2.2K, ../../CA+TgmoYFPbMX7DDuXnurJ1OOXoBpN4WbQg6SNubPPSNKtsRWiA@mail.gmail.com/2-example-execprocnodefirst-patch.diff)
download | inline diff:
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeead..536193e11d4 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -441,21 +441,37 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures
+ * (eg. x86). This relies on the assumption that ExecProcNode calls for
+ * a given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare
+ * not try to do this directly from CHECK_FOR_INTERRUPTS() because we
+ * don't really know what the executor state is at that point, but we
+ * assume that when entering a node the state will be sufficiently
+ * consistent that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan(node);
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2025-05-23 08:50 ` Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Atsushi Torikoshi @ 2025-05-23 08:50 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: torikoshia <[email protected]>; [email protected]; [email protected]; [email protected]
On Fri, May 23, 2025 at 12:08 AM Robert Haas <[email protected]> wrote:
Thanks for your review!
> On Tue, May 20, 2025 at 9:18 AM torikoshia <[email protected]> wrote:
> > I tackled this again and the attached patch removes ExecProcNodeOriginal
> > from Planstate.
> > Instead of adding a new field, this version builds the behavior into the
> > existing wrapper function, ExecProcNodeFirst().
> >
> > Since ExecProcNodeFirst() is already handling instrumentation-related
> > logic, the patch has maybe become a bit more complex to accommodate both
> > that and the new behavior.
> >
> > While it might make sense to introduce a more general mechanism that
> > allows for stacking an arbitrary number of wrappers around ExecProcNode,
> > I’m not sure it's possible or worth the added complexity—such layered
> > wrapping doesn't seem like something we typically need.
> >
> > What do you think?
>
> Hmm, I'm not convinced that this is correct. If
> GetProcessLogQueryPlanInterruptActive() is true but node->instrument
> is also non-NULL, your implementation of ExecProcNodeFirst() will
> handle the first but ignore the second.
In that case, the patch performs instrumentation during unwrapping --
specifically when executing UnwrapExecProcNodeWithExplain() -- so that
the query plan can still be logged for statements like "EXPLAIN
ANALYZE SELECT ..".
However, I admit this isn’t a good implementation: it’s hard to follow.
> Plus, I don't understand why
> ExecProcNodeFirst() does node->ExecProcNode = ExecProcNodeWithExplain
> instead of just directly doing the necessary work.
Indeed!
> It seems to me that
> this will result in the first call to ExecProcNode calling
> ExecProcNodeFirst to install ExecProcNodeWithExplain; and then the
> second call to ExecProcNode will call ExecProcNodeWithExplain which
> will actually do the work. But this seems unnecessary to me: I think
> it could just say if (GetProcessLogQueryPlanInterruptActive())
> LogQueryPlan(ps).
> Backing up a step, I think you've got a good idea here in thinking
> that we can probably reuse ExecProcNodeFirst for this purpose. It
> appears to me that it is always valid to reset a node's ExecProcNode
> pointer back to ExecProcNodeFirst. It might be unnecessary and cost
> some performance to do it when not required, but it is safe, because
> ExecProcNodeFirst will simply reset node->ExecProcNode and then call
> the appropriate function. So what we can do is just add the additional
> logic to check whether we need to print the query plan after the
> check_stack_depth() call and before the rest of the logic in the
> function. I've attached a sample patch to show what I have in mind.
Thanks for the idea and the sample patch!
Agreed. I’ll go ahead and implement a new patch based on this approach.
--
Atsushi Torikoshi
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
@ 2025-06-02 12:56 ` torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-06-02 12:56 UTC (permalink / raw)
To: Atsushi Torikoshi <[email protected]>; [email protected]; +Cc: [email protected]; [email protected]; [email protected]
On 2025-05-23 17:50, Atsushi Torikoshi wrote:
> Thanks for the idea and the sample patch!
> Agreed. I’ll go ahead and implement a new patch based on this approach.
Updated the patch.
Here are some notes:
As with the previous patches, this one wraps not only the currently
executing plan node but also recursively wraps the left, right, and
child nodes' ExecProcNode with ExecProcNodeFirst.
This is because modifying only the currently executing node caused
significant delays in plan logging when the left, right, or child nodes
took a long time to execute.
I observed the situation with the following query:
SELECT count(*) FROM pgbench_accounts a CROSS JOIN pgbench_accounts b;
Previous patches implemented unwrappers, but this one doesn’t.
This is because once the log is output,
GetProcessLogQueryPlanInterruptActive() returns false, so LogQueryPlan()
will no longer be called.
In the sample you previously provided, the LogQueryPlan function takes a
PlanState as an argument, but in this patch, it’s defined as void.
I made this change under the assumption that the plan can be obtained
from ActiveQueryDesc, and that PlanState is therefore unnecessary.
Please let me know if I’ve misunderstood anything.
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA GROUP CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v45-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.5K, ../../[email protected]/2-v45-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 3483da9204475fb9dc7bdd67d5db32510bd32ab4 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 2 Jun 2025 21:10:06 +0900
Subject: [PATCH v45] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcnodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Upon receiving a request to log the query plan, the ExecProcNode
functions of the current plan node, as well as its left, right,
and other child nodes, are wrapped with ExecProcNodeFirst, which
implements the logging mechanism. When the executor invokes any
of the wrapped nodes, the query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Squashed commit of the following:
---
contrib/auto_explain/auto_explain.c | 24 +--
doc/src/sgml/func.sgml | 57 ++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 195 +++++++++++++++++++
src/backend/commands/explain.c | 38 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 120 +++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 29 +++
src/include/commands/explain.h | 5 +
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 2 +-
src/include/storage/procsignal.h | 2 +
src/include/tcop/pquery.h | 1 -
src/test/regress/expected/misc_functions.out | 57 +++++-
src/test/regress/sql/misc_functions.sql | 45 ++++-
23 files changed, 573 insertions(+), 54 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 1f4badb492..6c4217c9d1 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -404,26 +405,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c67688cbf5..75a48a5744 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28684,6 +28684,29 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -28802,6 +28825,40 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+ Query Identifier: 8621255546560739680
+</screen>
+ When the target is executing nested statements(statements executed
+ inside a function), only the innermost query plan is logged.
+ Logging plan may take some time, as it occurs when the plan node is
+ executed. For example, when a query is running <function>pg_sleep</function>,
+ the plan will not be logged until the function execution completes.
+ Similarly, when a query is running under the extended query
+ protocol, the plan is logged only during the execute step.
+ <function>pg_log_query_plan()</function> may return <literal>true</literal>
+ even if no plan is actually logged, when the query is already about to
+ finish and the plan logging request comes too late.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 2e67e998ad..5df4dd0253 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -2932,6 +2933,12 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /*
+ * After abort, some elements of ActiveQueryDesc are freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ResetLogQueryPlanState();
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5324,6 +5331,12 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * After abort, some elements of ActiveQueryDesc are freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ResetLogQueryPlanState();
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 566f308e44..ec2e1c1fd9 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -775,6 +775,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c6..5e83907eb1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 0000000000..cfc13ce029
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,195 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+
+/* Currently executing query's QueryDesc */
+static QueryDesc *ActiveQueryDesc = NULL;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Clear pg_log_query_plan() related state during (sub)transaction abort.
+ */
+void
+ResetLogQueryPlanState(void)
+{
+ ActiveQueryDesc = NULL;
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * ActiveQueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check ActiveQueryDesc.
+ */
+ if (ActiveQueryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is finishing query execution and cannot log the plan.",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ }
+
+ ExplainStringAssemble(es, ActiveQueryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs corrent query plan if
+ * requested. This way ensures that EXPLAIN code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /* Wrap ExecProcNodes */
+ ExecSetExecProcNodeRecurse(ActiveQueryDesc->planstate);
+}
+
+/*
+ * Returns ActiveQueryDesc.
+ */
+QueryDesc *
+GetActiveQueryDesc(void)
+{
+ return ActiveQueryDesc;
+}
+
+/*
+ Set ActiveQueryDesc to queryDesc.
+ */
+void
+SetActiveQueryDesc(QueryDesc *queryDesc)
+{
+ ActiveQueryDesc = queryDesc;
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 7e2792ead7..a68958c1ad 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1084,6 +1084,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1815,7 +1846,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use instrumentation
+ * and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1823,7 +1857,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d3..4a64b8e9d0 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 0391798dd2..99727e5571 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -313,6 +314,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldActiveQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -325,6 +327,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save ActiveQueryDesc here to enable retrieval of the currently running
+ * queryDesc for nested queries.
+ */
+ oldActiveQueryDesc = GetActiveQueryDesc();
+ SetActiveQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -387,6 +396,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetActiveQueryDesc(oldActiveQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeea..e7749d15af 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -546,6 +564,88 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index a9bb540b55..2226ad0216 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -691,6 +692,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 2f8c3d5f91..6bdc7ba77a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/dynamic_explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3533,6 +3534,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a05..927ccda030 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d3d28a263f..e3bf1cb9cd 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8571,6 +8571,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 0000000000..be387a8cf9
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,29 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ResetLogQueryPlanState(void);
+extern void ResetProcessLogQueryPlanInterruptActive(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern QueryDesc *GetActiveQueryDesc(void);
+extern void SetActiveQueryDesc(QueryDesc *queryDesc);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3b122f79ed..6aa838764e 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -70,6 +70,8 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
@@ -80,5 +82,8 @@ extern void ExplainPrintJITSummary(struct ExplainState *es,
extern void ExplainQueryText(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(struct ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 32728f5d1a..fcef956396 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 104b059544..05a023940a 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -292,6 +292,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c..b26d7b919c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,7 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
-
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index afeeb1ca01..ea26242c86 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index fa3cc5f2df..7949abf76b 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -22,7 +22,6 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
extern PortalStrategy ChoosePortalStrategy(List *stmts);
extern List *FetchPortalTargetList(Portal portal);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3b2b9d860..1ac584fa7f 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -316,14 +316,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -337,8 +338,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -346,15 +347,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -363,8 +364,44 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+-- Note that we're using a slightly more complex query here because
+-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
+-- before it reaches the code path that actually outputs the plan.
+SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid())) result;
+ result
+--------
+ (t)
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 23792c4132..d8c141024e 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -109,39 +109,64 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+-- Note that we're using a slightly more complex query here because
+-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
+-- before it reaches the code path that actually outputs the plan.
+
+SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid())) result;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: fc32be3c941f9d98dd9f549153a5fcea6c3e9b8b
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-09-01 13:35 ` torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-09-01 13:35 UTC (permalink / raw)
To: Atsushi Torikoshi <[email protected]>; [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
On 2025-06-02 21:56, torikoshia wrote:
> On 2025-05-23 17:50, Atsushi Torikoshi wrote:
>> Thanks for the idea and the sample patch!
>> Agreed. I’ll go ahead and implement a new patch based on this
>> approach.
Rebased just because of doc refactoring.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
Attachments:
[text/x-diff] v46-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (36.6K, ../../[email protected]/2-v46-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 630c09643ee2787e96b4809878b432ff5390005d Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 1 Sep 2025 22:18:40 +0900
Subject: [PATCH v46] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcnodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Upon receiving a request to log the query plan, the ExecProcNode
functions of the current plan node, as well as its left, right,
and other child nodes, are wrapped with ExecProcNodeFirst, which
implements the logging mechanism. When the executor invokes any
of the wrapped nodes, the query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
---
contrib/auto_explain/auto_explain.c | 24 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 195 +++++++++++++++++++
src/backend/commands/explain.c | 38 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 120 +++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 29 +++
src/include/commands/explain.h | 5 +
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 2 +-
src/include/storage/procsignal.h | 2 +
src/include/tcop/pquery.h | 1 -
src/test/regress/expected/misc_functions.out | 57 +++++-
src/test/regress/sql/misc_functions.sql | 45 ++++-
23 files changed, 540 insertions(+), 54 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 1f4badb4928..6c4217c9d1f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -404,26 +405,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 57ff333159f..cda6e0d7fac 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b46e7e9c2a6..773ef89f1ed 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -2916,6 +2917,12 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /*
+ * After abort, some elements of ActiveQueryDesc are freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ResetLogQueryPlanState();
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5308,6 +5315,12 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * After abort, some elements of ActiveQueryDesc are freed. To avoid
+ * accessing them, reset ActiveQueryDesc here.
+ */
+ ResetLogQueryPlanState();
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 566f308e443..ec2e1c1fd9a 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -775,6 +775,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c60..5e83907eb17 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..cfc13ce0292
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,195 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+
+/* Currently executing query's QueryDesc */
+static QueryDesc *ActiveQueryDesc = NULL;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Clear pg_log_query_plan() related state during (sub)transaction abort.
+ */
+void
+ResetLogQueryPlanState(void)
+{
+ ActiveQueryDesc = NULL;
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * ActiveQueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check ActiveQueryDesc.
+ */
+ if (ActiveQueryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is finishing query execution and cannot log the plan.",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ }
+
+ ExplainStringAssemble(es, ActiveQueryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs corrent query plan if
+ * requested. This way ensures that EXPLAIN code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /* Wrap ExecProcNodes */
+ ExecSetExecProcNodeRecurse(ActiveQueryDesc->planstate);
+}
+
+/*
+ * Returns ActiveQueryDesc.
+ */
+QueryDesc *
+GetActiveQueryDesc(void)
+{
+ return ActiveQueryDesc;
+}
+
+/*
+ Set ActiveQueryDesc to queryDesc.
+ */
+void
+SetActiveQueryDesc(QueryDesc *queryDesc)
+{
+ ActiveQueryDesc = queryDesc;
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8345bc0264b..ded9c7aa856 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1084,6 +1084,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1815,7 +1846,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use instrumentation
+ * and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1823,7 +1857,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d32..4a64b8e9d09 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b8b9d2a85f7..98e81a87945 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -312,6 +313,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldActiveQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -324,6 +326,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save ActiveQueryDesc here to enable retrieval of the currently running
+ * queryDesc for nested queries.
+ */
+ oldActiveQueryDesc = GetActiveQueryDesc();
+ SetActiveQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -386,6 +395,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetActiveQueryDesc(oldActiveQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeead..e7749d15af8 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -546,6 +564,88 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 087821311cc..07fcac7f0bc 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -691,6 +692,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0cecd464902..f955b075ada 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/dynamic_explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3534,6 +3535,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..927ccda0307 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 118d6da1ace..c305ccd609a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8597,6 +8597,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..be387a8cf95
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,29 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ResetLogQueryPlanState(void);
+extern void ResetProcessLogQueryPlanInterruptActive(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern QueryDesc *GetActiveQueryDesc(void);
+extern void SetActiveQueryDesc(QueryDesc *queryDesc);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3b122f79ed8..6aa838764ee 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -70,6 +70,8 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
@@ -80,5 +82,8 @@ extern void ExplainPrintJITSummary(struct ExplainState *es,
extern void ExplainQueryText(struct ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(struct ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 32728f5d1a1..fcef9563960 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 10dcea037c3..60b1abc4869 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -294,6 +294,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..b26d7b919cb 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,7 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
-
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index afeeb1ca019..ea26242c868 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index fa3cc5f2dfc..7949abf76be 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -22,7 +22,6 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
extern PortalStrategy ChoosePortalStrategy(List *stmts);
extern List *FetchPortalTargetList(Portal portal);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3b2b9d8603..1ac584fa7f9 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -316,14 +316,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -337,8 +338,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -346,15 +347,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -363,8 +364,44 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+-- Note that we're using a slightly more complex query here because
+-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
+-- before it reaches the code path that actually outputs the plan.
+SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid())) result;
+ result
+--------
+ (t)
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 23792c4132a..d8c141024e1 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -109,39 +109,64 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+-- Note that we're using a slightly more complex query here because
+-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
+-- before it reaches the code path that actually outputs the plan.
+
+SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid())) result;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-09-09 17:59 ` Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2025-09-09 17:59 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On Mon, Sep 1, 2025 at 9:35 AM torikoshia <[email protected]> wrote:
> Rebased just because of doc refactoring.
I haven't had time to review this in a while -- sorry about that --
and I have only limited time now, but let me try to give you some
comments in the time that I do have.
I bet some users would really like a feature that allows them to get
the plans for their own running queries. They will be sad with the
limitation of this feature, that it only writes to the logs. We should
probably just live with that limitation, because doing anything else
seems a lot more complicated. Still, it's not great.
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is finishing query execution and cannot
log the plan.",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a
subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
The first of these messages ends with a period, which is contrary to
style guidelines. All of them do errhidestmt() and errhidecontext(),
which is an unusual choice. I don't see why it's appropriate here.
Wanting to see both the query and the query plan seems pretty
reasonable to me. The first two messages seem fairly unhelpful to me:
the user isn't going to understand the distinction between those two
states and it's unclear why we should give them that information. I'm
not sure if we should log a generic message in these kinds of cases or
log nothing at all, but I feel like this is too much technical
information. Also, I think that we don't normally put the PID of the
process that is performing an action in the primary error message,
because the user can include %p in log_line_prefix if they so desire.
But there's another, subtler point here, which is that I feel like "is
not running a query or a subtransaction is aborted" is pointing to a
design defect in the patch. When we enter an inner query, we switch
activeQueryDesc to point to the inner QueryDesc. When we abort a
subtransaction, instead of resetting it to the outer query's
QueryDesc, we reset it to NULL. I don't really think that's
acceptable. Consider this:
SELECT some_slow_function_that_uses_a_subtransaction_which_aborts(g)
FROM generate_series(1,1000000) g;
What's going to happen here is that for 99.9999% of the execution time
of this function, you can't print the query plan. And that won't be
because of any fundamental thing is preventing you from so doing -- it
will just be because the patch doesn't have the right bookkeeping. If
you added a QueryDesc * pointer to TransactionStateData, then
ExecutorRun could (1) save the current value of that pointer from the
topmost element of the transaction state stack, (2) update the pointer
value to the new QueryDesc, and (3) put the old pointer back at the
end. Then, you wouldn't need any cleanup in AbortSubTransaction, and
this value would always be right, even after an outer query continues
after an abort, because the subtransaction abort would pop the
transaction state stack, leaving the right thing on top.
The changes to miscadmin.h delete a blank line. They probably shouldn't.
The only change to pquery.h is to delete a blank line. That hunk needs
reverting.
+-- Note that we're using a slightly more complex query here because
+-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
+-- before it reaches the code path that actually outputs the plan.
+SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid())) result;
+ result
+--------
+ (t)
+(1 row)
I seriously doubt that this will be stable across the entire
buildfarm. You're going to need a different approach here.
Is there any real reason to rename regress_log_memory to
regress_log_function, vs. just introducing a separate regress_log_plan
role?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2025-09-16 13:30 ` torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-09-16 13:30 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On 2025-09-10 02:59, Robert Haas wrote:
> I haven't had time to review this in a while -- sorry about that --
> and I have only limited time now, but let me try to give you some
> comments in the time that I do have.
Thank you very much for taking the time to review this!
> I bet some users would really like a feature that allows them to get
> the plans for their own running queries. They will be sad with the
> limitation of this feature, that it only writes to the logs. We should
> probably just live with that limitation, because doing anything else
> seems a lot more complicated. Still, it's not great.
I agree.
Regarding output in a form other than the logs, I think the idea
proposed in [1] — making it possible to inspect memory contexts of other
backends via a view — would be a useful reference.
[1]
https://www.postgresql.org/message-id/flat/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw@mail....
> The first of these messages ends with a period, which is contrary to
> style guidelines.
Removed the period.
> All of them do errhidestmt() and errhidecontext(),
> which is an unusual choice. I don't see why it's appropriate here.
> Wanting to see both the query and the query plan seems pretty
> reasonable to me.
Removed errhidestmt() and errhidecontext().
> The first two messages seem fairly unhelpful to me:
> the user isn't going to understand the distinction between those two
> states and it's unclear why we should give them that information. I'm
> not sure if we should log a generic message in these kinds of cases or
> log nothing at all, but I feel like this is too much technical
> information.
I'm not sure if this is the best approach, but I changed them to log
nothing in these cases.
In addition to the reason you mentioned, I found that when
pg_log_query_plan() is executed repeatedly at very short intervals
(e.g., every 0.01 seconds), ereport() could lead to
ProcessLogQueryPlanInterrupt() being invoked again inside errfinish via
CFI(), which resulted in a stack overflow.
While this could be processed with check_stack_depth(), it doesn’t seem
worthwhile to use it just to emit a message of limited usefulness.
Additionally, there are other cases where the plan cannot be logged
(e.g., when the query is in a state which ExecProcNode will not be
invoked any further).
So instead, I added the following note to the documentation:
Note that depending on the execution state of the query,
it may not be possible to log the plan.
> Also, I think that we don't normally put the PID of the
> process that is performing an action in the primary error message,
> because the user can include %p in log_line_prefix if they so desire.
That makes sense, but
I had used the following message from pg_log_backend_memory_contexts()
as a reference
and put the PID in the message:
errmsg("logging memory contexts of PID %d", MyProcPid)));
Since both pg_log_query_plan() and pg_log_backend_memory_contexts()
output information about another other backend, it feels slightly odd
that their log messages would be inconsistent.
Perhaps should we consider changing pg_log_backend_memory_contexts() for
consistency as well?
> But there's another, subtler point here, which is that I feel like "is
> not running a query or a subtransaction is aborted" is pointing to a
> design defect in the patch. When we enter an inner query, we switch
> activeQueryDesc to point to the inner QueryDesc. When we abort a
> subtransaction, instead of resetting it to the outer query's
> QueryDesc, we reset it to NULL. I don't really think that's
> acceptable. Consider this:
>
> SELECT some_slow_function_that_uses_a_subtransaction_which_aborts(g)
> FROM generate_series(1,1000000) g;
>
> What's going to happen here is that for 99.9999% of the execution time
> of this function, you can't print the query plan. And that won't be
> because of any fundamental thing is preventing you from so doing -- it
> will just be because the patch doesn't have the right bookkeeping. If
> you added a QueryDesc * pointer to TransactionStateData, then
> ExecutorRun could (1) save the current value of that pointer from the
> topmost element of the transaction state stack, (2) update the pointer
> value to the new QueryDesc, and (3) put the old pointer back at the
> end.
Thank you for pointing out the issue and suggesting a fix.
Attached patch added a QueryDesc pointer to TransactionStateData and
updated it accordingly.
With this change, while running the query below, pg_log_query_plan() is
now able to output the plan for SELECT gen_subtxn(g) FROM
generate_series(1,1000000) g as below:
(session1)=# CREATE OR REPLACE FUNCTION gen_subtxn(i int)
RETURNS int LANGUAGE plpgsql AS $$
DECLARE
result int;
BEGIN
BEGIN
PERFORM 1 / 0;
EXCEPTION
WHEN division_by_zero THEN
result := -i;
END;
RETURN result;
END;
$$;
=# SELECT gen_subtxn(g) from generate_series(1,1000000) g;
(session2)=# SELECT pg_log_query_plan(pid) FROM pg_stat_activity WHERE
backend_type = 'client backend';
(session2)=# \watch 0.1
(in the log)
LOG: 00000: query and its plan running on the backend are:
Query Text: SELECT gen_subtxn(g) from generate_series(1,1000000) g;
Function Scan on pg_catalog.generate_series g (cost=0.00..260000.00
rows=1000000 width=4)
Output: gen_subtxn(g)
Function Call: generate_series(1, 1000000)
> Then, you wouldn't need any cleanup in AbortSubTransaction, and
> this value would always be right, even after an outer query continues
> after an abort, because the subtransaction abort would pop the
> transaction state stack, leaving the right thing on top.
It seems that simply making the above change is not sufficient.
In particular, when raising an error inside a subtransaction, I
sometimes observed segfaults:
BEGIN;
savepoint x;
CREATE TABLE koju (a INT UNIQUE);
INSERT INTO koju VALUES (1);
INSERT INTO koju VALUES (1); -- key duplicate error
Inspecting the crash showed that the argument *ps to
ExecSetExecProcNodeRecurse() was 0x7f7f7f....
(lldb) f 0
frame #0: 0x000000010102ec40
postgres`ExecSetExecProcNodeRecurse(ps=0x7f7f7f7f7f7f7f7f) at
execProcnode.c:601:30
600 {
-> 601 ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
This happens when the plan-logging function is called after an ERROR is
raised but before the transaction state stack is popped.
So in the attached patch, as in the previous version, I reset QueryDesc
to NULL during that interval.
I have also confirmed that even with this change, the above test (SELECT
gen_subtxn(g) from generate_series(1,1000000) g;) can still have its
plan logged successfully.
> The changes to miscadmin.h delete a blank line. They probably
> shouldn't.
Fixed.
> The only change to pquery.h is to delete a blank line. That hunk needs
> reverting.
Fixed.
> +-- Note that we're using a slightly more complex query here because
> +-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
> +-- before it reaches the code path that actually outputs the plan.
> +SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid()))
> result;
> + result
> +--------
> + (t)
> +(1 row)
>
> I seriously doubt that this will be stable across the entire
> buildfarm. You're going to need a different approach here.
Changed it to the following query:
WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
SELECT * FROM t;
Also since when the target query is using CTE, pg_log_query_plan() had
few chance to log the plan, attached patch added T_CteScan handling.
> Is there any real reason to rename regress_log_memory to
> regress_log_function, vs. just introducing a separate regress_log_plan
> role?
I don’t have any particular reason to insist on unifying them.
In the attached patch, I created a regress_log_plan role and used that
to run the same test.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
Attachments:
[text/x-diff] v47-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (34.5K, ../../[email protected]/2-v47-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 40acd5bf9ecbb386ba984a74552ac7393e6cc75a Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 16 Sep 2025 22:07:13 +0900
Subject: [PATCH v47] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
---
contrib/auto_explain/auto_explain.c | 24 +--
doc/src/sgml/func/func-admin.sgml | 28 +++
src/backend/access/transam/xact.c | 30 ++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 173 +++++++++++++++++++
src/backend/commands/explain.c | 38 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 123 +++++++++++--
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 27 +++
src/include/commands/explain.h | 9 +-
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/regress/expected/misc_functions.out | 43 +++++
src/test/regress/sql/misc_functions.sql | 32 ++++
23 files changed, 538 insertions(+), 34 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 1f4badb4928..6c4217c9d1f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -404,26 +405,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 57ff333159f..1dc19bbe3b4 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,34 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ Note that depending on the execution state of the query,
+ it may not be possible to log the plan.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b46e7e9c2a6..8c2e7d5061f 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -215,6 +216,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -248,6 +250,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -933,6 +936,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2916,6 +2940,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset state for logging current query plan. */
+ ResetLogQueryPlanState();
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5308,6 +5335,9 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /* Reset state for logging current query plan. */
+ ResetLogQueryPlanState();
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 566f308e443..ec2e1c1fd9a 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -775,6 +775,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c60..5e83907eb17 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..a60cc9bea2e
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,173 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Clear pg_log_query_plan() related state during (sub)transaction abort.
+ */
+void
+ResetLogQueryPlanState(void)
+{
+ /*
+ * After abort, some elements of current QueryDesc are freed. To avoid
+ * accessing them, reset it.
+ */
+ SetCurrentQueryDesc(NULL);
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on the backend are:\n%s",
+ es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /* Wrap ExecProcNodes with ExecProcNodeFirst */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8345bc0264b..556779bafac 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1084,6 +1084,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1815,7 +1846,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1823,7 +1857,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d32..4a64b8e9d09 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index ff12e2e1364..b6f72d78ae6 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -312,6 +313,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -324,6 +326,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -386,6 +395,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeead..bf1e26b7af1 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -546,6 +564,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 087821311cc..07fcac7f0bc 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -691,6 +692,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index d356830f756..c5a5cd47c8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/dynamic_explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3538,6 +3539,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..927ccda0307 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index 4528e51829e..3c6e3b7f2ce 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern void CommandCounterIncrement(void);
extern void ForceSyncCommit(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 03e82d28c87..6b5ab18bebd 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8609,6 +8609,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..83c158731a1
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,27 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ResetLogQueryPlanState(void);
+extern void ResetProcessLogQueryPlanInterruptActive(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 6e51d50efc7..0dc160fa7b4 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -70,8 +70,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
-extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
-extern void ExplainPrintTriggers(ExplainState *es,
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
extern void ExplainPrintJITSummary(ExplainState *es,
@@ -80,5 +82,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index ba073b86918..bf40fdabff5 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 3248e78cd28..c66df11e71a 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -295,6 +295,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..22b945e918c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index afeeb1ca019..ea26242c868 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3b2b9d8603..c2cc2609e94 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -366,6 +366,49 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 23792c4132a..88be4b6ef2a 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -143,6 +143,38 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-09-16 15:05 ` Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2025-09-16 15:05 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On Tue, Sep 16, 2025 at 9:30 AM torikoshia <[email protected]> wrote:
> I'm not sure if this is the best approach, but I changed them to log
> nothing in these cases.
> In addition to the reason you mentioned, I found that when
> pg_log_query_plan() is executed repeatedly at very short intervals
> (e.g., every 0.01 seconds), ereport() could lead to
> ProcessLogQueryPlanInterrupt() being invoked again inside errfinish via
> CFI(), which resulted in a stack overflow.
> While this could be processed with check_stack_depth(), it doesn’t seem
> worthwhile to use it just to emit a message of limited usefulness.
Yes, it seems as though we should set a flag to prevent reentrancy
here -- if we are already in the code path where we log the query
plan, we shouldn't accept an interrupt telling us to log the query
plan. I haven't looked at the updated patch so maybe that's not
necessary for some reason, but in general reentrancy is a concern with
features of this type.
> Additionally, there are other cases where the plan cannot be logged
> (e.g., when the query is in a state which ExecProcNode will not be
> invoked any further).
> So instead, I added the following note to the documentation:
>
> Note that depending on the execution state of the query,
> it may not be possible to log the plan.
This seems to me to be too vague to be useful. I expect readers to
read this to mean "sometimes this feature may not work." But that
seems too pessimistic if the only case in which it doesn't work is
when we are the very tail end of the query and it was just about to
stop running, we probably don't need to document anything, as it will
happen rarely and will look about the same as if the query finished
very slightly earlier and we just missed it. If there are more cases
than that in which this feature won't work, we should talk about
those, and maybe fix them.
> > Also, I think that we don't normally put the PID of the
> > process that is performing an action in the primary error message,
> > because the user can include %p in log_line_prefix if they so desire.
>
> That makes sense, but
> I had used the following message from pg_log_backend_memory_contexts()
> as a reference
> and put the PID in the message:
>
> errmsg("logging memory contexts of PID %d", MyProcPid)));
>
> Since both pg_log_query_plan() and pg_log_backend_memory_contexts()
> output information about another other backend, it feels slightly odd
> that their log messages would be inconsistent.
> Perhaps should we consider changing pg_log_backend_memory_contexts() for
> consistency as well?
Yeah, maybe. I don't know what the reason behind the unusual framing
of that message is, so perhaps there is an argument for being
consistent with it, rather than changing it. However, it looks unusual
to me.
> With this change, while running the query below, pg_log_query_plan() is
> now able to output the plan for SELECT gen_subtxn(g) FROM
> generate_series(1,1000000) g as below:
Excellent!
> This happens when the plan-logging function is called after an ERROR is
> raised but before the transaction state stack is popped.
> So in the attached patch, as in the previous version, I reset QueryDesc
> to NULL during that interval.
I think a better fix would be to dump nothing when the current
(sub)transaction is (sub)aborted. If you don't do that, then you're
hoping that you can manage to set QueryDesc to null quickly enough
after the transaction is no longer in a valid state, which does not
seem like a great way to make things robust.
> > +-- Note that we're using a slightly more complex query here because
> > +-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
> > +-- before it reaches the code path that actually outputs the plan.
> > +SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid()))
> > result;
> > + result
> > +--------
> > + (t)
> > +(1 row)
> >
> > I seriously doubt that this will be stable across the entire
> > buildfarm. You're going to need a different approach here.
>
> Changed it to the following query:
>
> WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
> SELECT * FROM t;
I don't see why that wouldn't have a race condition?
> Also since when the target query is using CTE, pg_log_query_plan() had
> few chance to log the plan, attached patch added T_CteScan handling.
Shouldn't all node types be handled equally?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2025-09-19 12:42 ` torikoshia <[email protected]>
2025-09-19 17:03 ` Re: RFC: Logging plan of the running query Rafael Thofehrn Castro <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: torikoshia @ 2025-09-19 12:42 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On Wed, Sep 17, 2025 at 12:06 AM Robert Haas <[email protected]>
wrote:
Thanks for the comments!
> On Tue, Sep 16, 2025 at 9:30 AM torikoshia <[email protected]>
> wrote:
> > I'm not sure if this is the best approach, but I changed them to log
> > nothing in these cases.
> > In addition to the reason you mentioned, I found that when
> > pg_log_query_plan() is executed repeatedly at very short intervals
> > (e.g., every 0.01 seconds), ereport() could lead to
> > ProcessLogQueryPlanInterrupt() being invoked again inside errfinish via
> > CFI(), which resulted in a stack overflow.
> > While this could be processed with check_stack_depth(), it doesn’t seem
> > worthwhile to use it just to emit a message of limited usefulness.
>
> Yes, it seems as though we should set a flag to prevent reentrancy
> here -- if we are already in the code path where we log the query
> plan, we shouldn't accept an interrupt telling us to log the query
> plan. I haven't looked at the updated patch so maybe that's not
> necessary for some reason, but in general reentrancy is a concern with
> features of this type.
Agreed. In the attached patch added a flag.
> > Additionally, there are other cases where the plan cannot be logged
> > (e.g., when the query is in a state which ExecProcNode will not be
> > invoked any further).
> > So instead, I added the following note to the documentation:
> >
> > Note that depending on the execution state of the query,
> > it may not be possible to log the plan.
>
> This seems to me to be too vague to be useful. I expect readers to
> read this to mean "sometimes this feature may not work." But that
> seems too pessimistic if the only case in which it doesn't work is
> when we are the very tail end of the query and it was just about to
> stop running, we probably don't need to document anything, as it will
> happen rarely and will look about the same as if the query finished
> very slightly earlier and we just missed it. If there are more cases
> than that in which this feature won't work, we should talk about
> those, and maybe fix them.
As you described, I think it's a rare case to have no chance to log
plan, removed the note.
> > > Also, I think that we don't normally put the PID of the
> > > process that is performing an action in the primary error message,
> > > because the user can include %p in log_line_prefix if they so desire.
> >
> > That makes sense, but
> > I had used the following message from pg_log_backend_memory_contexts()
> > as a reference
> > and put the PID in the message:
> >
> > errmsg("logging memory contexts of PID %d", MyProcPid)));
> >
> > Since both pg_log_query_plan() and pg_log_backend_memory_contexts()
> > output information about another other backend, it feels slightly odd
> > that their log messages would be inconsistent.
> > Perhaps should we consider changing pg_log_backend_memory_contexts() for
> > consistency as well?
>
> Yeah, maybe. I don't know what the reason behind the unusual framing
> of that message is, so perhaps there is an argument for being
> consistent with it, rather than changing it. However, it looks unusual
> to me.
Looking back at the discussion on the development of
pg_log_backend_memory_contexts(), it seems that I had already proposed
outputting the PID in the log message from the very beginning [1].
Originally, I had suggested making the memory context available directly
as a function result rather than logging it, but we gave up on that idea
and switched to logging, so I think the PID output may be a leftover
from that change.
[1]
https://www.postgresql.org/message-id/70ae4b79eb8b0dcf42161c80a00e3f22%40oss.nttdata.com
It’s possible that users who don’t include %p in log_line_prefix could
have trouble identifying which PID’s memory context is being logged when
pg_log_backend_memory_contexts() is called concurrently for multiple
processes.
However, I suspect that few users run without %p in log_line_prefix.
So I think it would be fine to remove the PID from
pg_log_backend_memory_contexts()’s log message when this patch is
merged.
> > With this change, while running the query below, pg_log_query_plan() is
> > now able to output the plan for SELECT gen_subtxn(g) FROM
> > generate_series(1,1000000) g as below:
>
> Excellent!
>
> > This happens when the plan-logging function is called after an ERROR is
> > raised but before the transaction state stack is popped.
> > So in the attached patch, as in the previous version, I reset QueryDesc
> > to NULL during that interval.
>
> I think a better fix would be to dump nothing when the current
> (sub)transaction is (sub)aborted. If you don't do that, then you're
> hoping that you can manage to set QueryDesc to null quickly enough
> after the transaction is no longer in a valid state, which does not
> seem like a great way to make things robust.
To detect “when the current (sub)transaction is (sub)aborted,” attached
patch uses IsTransactionState().
With that, the logging code is not executed when the transaction state
is anything other than TRANS_INPROGRESS, and in some cases this works as
expected.
On the other hand, when calling pg_log_query_plan() repeatedly during
make installcheck, I still encountered cases where segfaults occurred.
Specifically, this seems to happen when, after a (sub)abort, the next
query starts in the same session and a CFI() occurs after
StartTransaction() has been executed but before Executor() runs. In
that case, the transaction state has already been changed to
TRANS_INPROGRESS by StartTransaction(), but the QueryDesc from the
aborted transaction is still present, which causes the problem.
To address this, attached patch initializes QueryDesc within
CleanupTransaction().
Since this function already resets various members of
CurrentTransactionState, I feel it’s not so unreasonable to initialize
QueryDesc there as well.
Does this approach make sense, or is it problematic?
> > > +-- Note that we're using a slightly more complex query here because
> > > +-- a simple 'SELECT pg_log_query_plan(pg_backend_pid())' would finish
> > > +-- before it reaches the code path that actually outputs the plan.
> > > +SELECT result FROM (SELECT pg_log_query_plan(pg_backend_pid()))
> > > result;
> > > + result
> > > +--------
> > > + (t)
> > > +(1 row)
> > >
> > > I seriously doubt that this will be stable across the entire
> > > buildfarm. You're going to need a different approach here.
> >
> > Changed it to the following query:
> >
> > WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
> > SELECT * FROM t;
>
> I don't see why that wouldn't have a race condition?
The idea is that (1) pg_log_query_plan() in the WITH clause wraps
ExecProcNode, and then (2) SELECT * FROM t invokes ExecProcNode, which
causes the plan to be logged. I think that’s what currently happens on
HEAD.
When you mention a "race condition", do you mean that if the timing of
the wrapping in step (1) -- that is, when CFI() is called -- changes in
the future, then the logging might not work?
>> few chance to log the plan, attached patch added T_CteScan handling.
>
> Shouldn't all node types be handled equally?
IIUC the node types that would need to be handled in the same way are
those PlanState subclasses that satisfy both of the following:
- Structs in execnodes.h whose first field is a PlanState, and
- Subclasses that also contain another PlanState as a member in addition
to the first field.
If that’s the case, it seems to me that we already have the full list.
BTW I haven’t looked into this in detail yet, but I’m a little curious
whether the absence of T_CteScan in functions like
planstate_tree_walker_impl() is intentional.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
Attachments:
[text/x-diff] v48-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (34.2K, ../../[email protected]/2-v48-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 37d9e93b0dbfd5c662827160bfe85e4e8e3fd76b Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Fri, 19 Sep 2025 21:29:51 +0900
Subject: [PATCH v48] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Remove the PID from the log output of
pg_log_backend_memory_contexts() to match this patch.
---
contrib/auto_explain/auto_explain.c | 24 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 29 ++-
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 181 +++++++++++++++++++
src/backend/commands/explain.c | 38 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 123 ++++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 25 +++
src/include/commands/explain.h | 9 +-
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/regress/expected/misc_functions.out | 43 +++++
src/test/regress/sql/misc_functions.sql | 32 ++++
23 files changed, 537 insertions(+), 36 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 1f4badb4928..6c4217c9d1f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -404,26 +405,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 1b465bc8ba7..66bfb5ab4df 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b46e7e9c2a6..b9b374b9911 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -215,6 +216,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -248,6 +250,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -933,6 +936,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -3058,10 +3082,11 @@ CleanupTransaction(void)
nParallelCurrentXids = 0;
/*
- * done with abort processing, set current transaction state back to
- * default
+ * done with abort processing, set current transaction state and QueryDesc
+ * back to default
*/
s->state = TRANS_DEFAULT;
+ s->queryDesc = NULL;
}
/*
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 2d946d6d9e9..67beafcc82c 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -782,6 +782,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c60..5e83907eb17 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..d68203fe8aa
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,181 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+
+static bool isProcessingLogQueryPlan = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on the backend are:\n%s",
+ es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc;
+
+ /* Prevent re-entrant */
+ if (isProcessingLogQueryPlan)
+ return;
+
+ isProcessingLogQueryPlan = true;
+
+ /* Cannot log query plan outside a transaction */
+ if (!IsTransactionState())
+ {
+ isProcessingLogQueryPlan = false;
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ isProcessingLogQueryPlan = false;
+
+ return;
+ }
+
+ /* Wrap ExecProcNodes with ExecProcNodeFirst */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+
+ isProcessingLogQueryPlan = false;
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8345bc0264b..556779bafac 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1084,6 +1084,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1815,7 +1846,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1823,7 +1857,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d32..4a64b8e9d09 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 831c55ce787..df5b55ae64d 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -312,6 +313,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -324,6 +326,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -386,6 +395,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeead..bf1e26b7af1 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -546,6 +564,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 087821311cc..07fcac7f0bc 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -691,6 +692,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index d356830f756..c5a5cd47c8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/dynamic_explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3538,6 +3539,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..927ccda0307 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index 4528e51829e..3c6e3b7f2ce 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern void CommandCounterIncrement(void);
extern void ForceSyncCommit(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..6cbfa0af91f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8609,6 +8609,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..83a9e27deef
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,25 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 6e51d50efc7..0dc160fa7b4 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -70,8 +70,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
-extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
-extern void ExplainPrintTriggers(ExplainState *es,
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
extern void ExplainPrintJITSummary(ExplainState *es,
@@ -80,5 +82,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index ba073b86918..bf40fdabff5 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 3248e78cd28..c66df11e71a 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -295,6 +295,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..22b945e918c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index afeeb1ca019..ea26242c868 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 36164a99c83..67adc601a9f 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -366,6 +366,49 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 23792c4132a..88be4b6ef2a 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -143,6 +143,38 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-09-19 17:03 ` Rafael Thofehrn Castro <[email protected]>
2025-09-20 03:21 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Rafael Thofehrn Castro @ 2025-09-19 17:03 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Robert Haas <[email protected]>; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
Hi folks,
apologies for not replying earlier. Thanks torikoshia for having
reviewed the related patch I sent that includes instrumentation.
It makes total sense to focus on this one first.
Been thinking about the current strategy of having to iterate through
the execution tree to add the custom ExecProcNode to avoid logging
the plan in CHECK_FOR_INTERRUPTS(). I see you folks are still
discussing all the nuances in the recursive tree traversal function.
Taking a step back and proposing a different approach, have we thought
about logging the query plan in the regions of the code related to the
query executor, NEXT to CHECK_FOR_INTERRUPTS calls instead of IN
that function?
For example, in this part of executor/nodeHashjoin.c:
for (;;)
{
/*
* It's possible to iterate this loop many times before returning a
* tuple, in some pathological cases such as needing to move much of
* the current batch to a later batch. So let's check for interrupts
* each time through.
*/
CHECK_FOR_INTERRUPTS();
We replace CHECK_FOR_INTERRUPTS() for a new function that does
query plan logging logic + CHECK_FOR_INTERRUPTS().
Would that be considered a safe operation given that we would always be
in the query execution context? The current ExecProcNode wrapper strategy
logs the query plan in ExecProcNodeFirst(). That function will then call
the real ExecProcNode, which points to one of the functions that contain
the CHECK_FOR_INTERRUPTS in the first place. I don't see much difference
in terms of logging safety between the two approaches, but maybe there
is :)
Rafael.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-19 17:03 ` Re: RFC: Logging plan of the running query Rafael Thofehrn Castro <[email protected]>
@ 2025-09-20 03:21 ` Atsushi Torikoshi <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: Atsushi Torikoshi @ 2025-09-20 03:21 UTC (permalink / raw)
To: Rafael Thofehrn Castro <[email protected]>; +Cc: torikoshia <[email protected]>; Robert Haas <[email protected]>; [email protected]; [email protected]; [email protected]
On Sat, Sep 20, 2025 at 2:04 AM Rafael Thofehrn Castro
<[email protected]> wrote:
> apologies for not replying earlier. Thanks torikoshia for having
> reviewed the related patch I sent that includes instrumentation.
> It makes total sense to focus on this one first.
Thank you as well for agreeing on the direction for how to proceed!
> Been thinking about the current strategy of having to iterate through
> the execution tree to add the custom ExecProcNode to avoid logging
> the plan in CHECK_FOR_INTERRUPTS(). I see you folks are still
> discussing all the nuances in the recursive tree traversal function.
>
> Taking a step back and proposing a different approach, have we thought
> about logging the query plan in the regions of the code related to the
> query executor, NEXT to CHECK_FOR_INTERRUPTS calls instead of IN
> that function?
As a related discussion, there was an idea of splitting
CHECK_FOR_INTERRUPTS() into two variants: one for cases where it’s
safe to perform operations like plan logging, and one where it isn’t:
https://www.postgresql.org/message-id/CAAaqYe-gMkdL%3DM4v47%3DH0F3%2B-zi2qL9zFqAv3QsizkRjFiQR0w%40ma...
However, as I mentioned later in this thread, there were some issues,
so the current discussion has focused on the node-wrapping approach
instead.
> For example, in this part of executor/nodeHashjoin.c:
>
> for (;;)
> {
> /*
> * It's possible to iterate this loop many times before returning a
> * tuple, in some pathological cases such as needing to move much of
> * the current batch to a later batch. So let's check for interrupts
> * each time through.
> */
> CHECK_FOR_INTERRUPTS();
>
> We replace CHECK_FOR_INTERRUPTS() for a new function that does
> query plan logging logic + CHECK_FOR_INTERRUPTS().
>
> Would that be considered a safe operation given that we would always be
> in the query execution context? The current ExecProcNode wrapper strategy
> logs the query plan in ExecProcNodeFirst(). That function will then call
> the real ExecProcNode, which points to one of the functions that contain
> the CHECK_FOR_INTERRUPTS in the first place. I don't see much difference
> in terms of logging safety between the two approaches, but maybe there
> is :)
IIUC, the differences between the two approaches are:
(1) Timing of plan output
Current patch: The plan is logged the first time each node’s
ExecProcNode() begins execution.
Your idea: The plan is logged at each “NEXT to CHECK_FOR_INTERRUPTS()”
point inside the node.
(2) Number of checks for whether plan output is needed
Current patch: Once after logging plan is requested
Your idea: Every time execution reaches a “NEXT to
CHECK_FOR_INTERRUPTS()” point inside the node.
At least Robert and I believe that the first execution of each node’s
ExecProcNode() represents a sufficiently consistent state for plan
output, as noted in the patch comments:
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
Investigating consistency at each of those points would not be easy,
and given the variety of node types, while not impossible, it would be
very difficult.
As for point (2), it’s not directly about safety, but the idea could
introduce performance overhead due to the repeated checks.
Regards,
Atsushi Torikoshi
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-09-24 16:34 ` Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Robert Haas @ 2025-09-24 16:34 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On Fri, Sep 19, 2025 at 8:42 AM torikoshia <[email protected]> wrote:
> > Yes, it seems as though we should set a flag to prevent reentrancy
> > here -- if we are already in the code path where we log the query
> > plan, we shouldn't accept an interrupt telling us to log the query
> > plan. I haven't looked at the updated patch so maybe that's not
> > necessary for some reason, but in general reentrancy is a concern with
> > features of this type.
>
> Agreed. In the attached patch added a flag.
This doesn't look safe. If we error out of the section where the flag
is set to true, it will remain true forever.
> So I think it would be fine to remove the PID from
> pg_log_backend_memory_contexts()’s log message when this patch is
> merged.
Let's leave the PID in there for now for consistency with the existing
message. We can discuss changing it at another time.
> To detect “when the current (sub)transaction is (sub)aborted,” attached
> patch uses IsTransactionState().
> With that, the logging code is not executed when the transaction state
> is anything other than TRANS_INPROGRESS, and in some cases this works as
> expected.
> On the other hand, when calling pg_log_query_plan() repeatedly during
> make installcheck, I still encountered cases where segfaults occurred.
>
> Specifically, this seems to happen when, after a (sub)abort, the next
> query starts in the same session and a CFI() occurs after
> StartTransaction() has been executed but before Executor() runs. In
> that case, the transaction state has already been changed to
> TRANS_INPROGRESS by StartTransaction(), but the QueryDesc from the
> aborted transaction is still present, which causes the problem.
>
> To address this, attached patch initializes QueryDesc within
> CleanupTransaction().
> Since this function already resets various members of
> CurrentTransactionState, I feel it’s not so unreasonable to initialize
> QueryDesc there as well.
> Does this approach make sense, or is it problematic?
Hmm, I think it looks quite odd. I believe it's quite intentional that
the very last thing that CleanupTransaction does is reset s->state, so
I don't think we should be doing this after that. But also, this leads
to an odd inconsistency between CleanupTransaction and
CleanupSubTransaction. What I'm wondering is whether we should instead
reset s->queryDesc inside AbortTransaction() and
AbortSubTransaction(). Perhaps if we do that, then we don't need an
InTransactionState() test elsewhere. What do you think?
\> > > Changed it to the following query:
> > >
> > > WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
> > > SELECT * FROM t;
> >
> > I don't see why that wouldn't have a race condition?
>
> The idea is that (1) pg_log_query_plan() in the WITH clause wraps
> ExecProcNode, and then (2) SELECT * FROM t invokes ExecProcNode, which
> causes the plan to be logged. I think that’s what currently happens on
> HEAD.
> When you mention a "race condition", do you mean that if the timing of
> the wrapping in step (1) -- that is, when CFI() is called -- changes in
> the future, then the logging might not work?
I don't think we're guaranteed that the signal is delivered instantly,
so it seems possible to me that (1) would happen after (2).
> BTW I haven’t looked into this in detail yet, but I’m a little curious
> whether the absence of T_CteScan in functions like
> planstate_tree_walker_impl() is intentional.
I don't think that function needs any special handling for T_CteScan.
planstate_tree_walker_impl() and other functions need special handling
for cases where a node has children that are not in the lefttree,
righttree, initPlan list, or subPlan list; but a CteScan has no extra
Plan pointer:
typedef struct CteScan
{
Scan scan;
/* ID of init SubPlan for CTE */
int ctePlanId;
/* ID of Param representing CTE output */
int cteParam;
} CteScan;
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2025-10-01 09:11 ` torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-10-01 09:11 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On 2025-09-25 01:34, Robert Haas wrote:
> On Fri, Sep 19, 2025 at 8:42 AM torikoshia <[email protected]>
> wrote:
>> > Yes, it seems as though we should set a flag to prevent reentrancy
>> > here -- if we are already in the code path where we log the query
>> > plan, we shouldn't accept an interrupt telling us to log the query
>> > plan. I haven't looked at the updated patch so maybe that's not
>> > necessary for some reason, but in general reentrancy is a concern with
>> > features of this type.
>>
>> Agreed. In the attached patch added a flag.
>
> This doesn't look safe. If we error out of the section where the flag
> is set to true, it will remain true forever.
Ugh.. Updated the patch to reset the flag even when an error occurs,
using PG_FINALLY(), similar to [1].
[1]
https://www.postgresql.org/message-id/27da56de-17c4-4af2-8032-3c58ae0c7b00%40oss.nttdata.com
>> So I think it would be fine to remove the PID from
>> pg_log_backend_memory_contexts()’s log message when this patch is
>> merged.
>
> Let's leave the PID in there for now for consistency with the existing
> message. We can discuss changing it at another time.
Agreed.
>> To detect “when the current (sub)transaction is (sub)aborted,”
>> attached
>> patch uses IsTransactionState().
>> With that, the logging code is not executed when the transaction state
>> is anything other than TRANS_INPROGRESS, and in some cases this works
>> as
>> expected.
>> On the other hand, when calling pg_log_query_plan() repeatedly during
>> make installcheck, I still encountered cases where segfaults occurred.
>>
>> Specifically, this seems to happen when, after a (sub)abort, the next
>> query starts in the same session and a CFI() occurs after
>> StartTransaction() has been executed but before Executor() runs. In
>> that case, the transaction state has already been changed to
>> TRANS_INPROGRESS by StartTransaction(), but the QueryDesc from the
>> aborted transaction is still present, which causes the problem.
>>
>> To address this, attached patch initializes QueryDesc within
>> CleanupTransaction().
>> Since this function already resets various members of
>> CurrentTransactionState, I feel it’s not so unreasonable to initialize
>> QueryDesc there as well.
>> Does this approach make sense, or is it problematic?
>
> Hmm, I think it looks quite odd. I believe it's quite intentional that
> the very last thing that CleanupTransaction does is reset s->state, so
> I don't think we should be doing this after that. But also, this leads
> to an odd inconsistency between CleanupTransaction and
> CleanupSubTransaction.
Make sense.
> What I'm wondering is whether we should instead
> reset s->queryDesc inside AbortTransaction() and
> AbortSubTransaction(). Perhaps if we do that, then we don't need an
> InTransactionState() test elsewhere. What do you think?
Resetting QueryDesc in AbortTransaction() and AbortSubTransaction() was
what we did in an earlier version of the patch (before v46), so that
doesn’t feel strange to me.
Updated the patch and confirmed that we can still log the parent’s plan
in cases where subtransactions continuously abort, following the same
steps as [2].
[2]
https://www.postgresql.org/message-id/6143f00af4bfdba95a85cf866f4acb41%40oss.nttdata.com
> \> > > Changed it to the following query:
>> > >
>> > > WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
>> > > SELECT * FROM t;
>> >
>> > I don't see why that wouldn't have a race condition?
>>
>> The idea is that (1) pg_log_query_plan() in the WITH clause wraps
>> ExecProcNode, and then (2) SELECT * FROM t invokes ExecProcNode, which
>> causes the plan to be logged. I think that’s what currently happens
>> on
>> HEAD.
>> When you mention a "race condition", do you mean that if the timing of
>> the wrapping in step (1) -- that is, when CFI() is called -- changes
>> in
>> the future, then the logging might not work?
>
> I don't think we're guaranteed that the signal is delivered instantly,
> so it seems possible to me that (1) would happen after (2).
True, but I haven’t found a reliable way to trigger actual logging
within a single backend query.
I was also considering using an isolation test and injection points,
like in the attached PoC patch. The main steps are:
In session1, set an injection point to wait during query execution.
In session1, run a query that waits at the injection point.
In session2, call pg_log_query_plan().
In session2, wake up session1.
However, as you pointed out, since the signals are not guaranteed to be
delivered instantly, I’m worried that the query could complete before
the signal is delivered, causing test instability.
Additionally, for a similar function that logs information via signal
delivery, pg_log_backend_memory_contexts(), the existing tests only
verify that the function succeeds and that permission checks work as
described in the comment below; I'm a bit concerned that this might be
overkill.
-- pg_log_backend_memory_contexts()
--
-- Memory contexts are logged and they are not returned to the
function.
-- Furthermore, their contents can vary depending on the timing.
However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
If we align with that test strategy, just executing SELECT
pg_log_query_plan(pg_backend_pid()) and verifying that it succeeds would
be enough.
However, unlike pg_log_backend_memory_contexts(), this function doesn’t
log anything for this query. I’m not sure if that’s acceptable.
I’d appreciate any thoughts or suggestions on what kind of test coverage
would be appropriate here.
>> BTW I haven’t looked into this in detail yet, but I’m a little curious
>> whether the absence of T_CteScan in functions like
>> planstate_tree_walker_impl() is intentional.
>
> I don't think that function needs any special handling for T_CteScan.
> planstate_tree_walker_impl() and other functions need special handling
> for cases where a node has children that are not in the lefttree,
> righttree, initPlan list, or subPlan list; but a CteScan has no extra
> Plan pointer:
>
> typedef struct CteScan
> {
> Scan scan;
> /* ID of init SubPlan for CTE */
> int ctePlanId;
> /* ID of Param representing CTE output */
> int cteParam;
> } CteScan;
Thanks for looking into this! understood.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
Attachments:
[text/x-diff] v49-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (34.9K, ../../[email protected]/2-v49-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 3089479df3bd3c3e4d996b1e20d4ebfd67aa82c4 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 1 Oct 2025 17:50:28 +0900
Subject: [PATCH v49] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 24 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 34 ++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 183 +++++++++++++++++++
src/backend/commands/explain.c | 38 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 123 ++++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 25 +++
src/include/commands/explain.h | 9 +-
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/regress/expected/misc_functions.out | 42 +++++
src/test/regress/sql/misc_functions.sql | 31 ++++
23 files changed, 544 insertions(+), 34 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 1f4badb4928..6c4217c9d1f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -404,26 +405,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 1b465bc8ba7..66bfb5ab4df 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 2cf3d4e92b7..56c3c196f45 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -215,6 +216,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -248,6 +250,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -933,6 +936,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2916,6 +2940,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current query plan state used for logging. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5308,6 +5335,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current query plan state used for logging. Note that even after
+ * this reset, it's still possible to obtain the parent transaction's
+ * query plans, since they are preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 2d946d6d9e9..67beafcc82c 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -782,6 +782,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c60..5e83907eb17 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..b21a51aafe4
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,183 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly and
+ * rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 207f86f1d39..92f02260e38 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1085,6 +1085,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1820,7 +1851,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1828,7 +1862,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d32..4a64b8e9d09 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 831c55ce787..df5b55ae64d 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -312,6 +313,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -324,6 +326,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -386,6 +395,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeead..bf1e26b7af1 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -546,6 +564,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 087821311cc..07fcac7f0bc 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -691,6 +692,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index d356830f756..c5a5cd47c8f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/dynamic_explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3538,6 +3539,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..927ccda0307 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index 4528e51829e..3c6e3b7f2ce 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern void CommandCounterIncrement(void);
extern void ForceSyncCommit(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..6cbfa0af91f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8609,6 +8609,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..83a9e27deef
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,25 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 6e51d50efc7..0dc160fa7b4 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -70,8 +70,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
-extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
-extern void ExplainPrintTriggers(ExplainState *es,
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
extern void ExplainPrintJITSummary(ExplainState *es,
@@ -80,5 +82,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index ba073b86918..bf40fdabff5 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 3248e78cd28..c66df11e71a 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -295,6 +295,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..22b945e918c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index afeeb1ca019..ea26242c868 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 36164a99c83..53fef3f9ee3 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -366,6 +366,48 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 23792c4132a..693b54b8316 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -143,6 +143,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+
+WITH t AS MATERIALIZED (SELECT pg_log_query_plan(pg_backend_pid()))
+ SELECT * FROM t;
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.48.1
[text/x-diff] PoC-injection-point-test-for-pg_log_query_plan.txt (3.5K, ../../[email protected]/3-PoC-injection-point-test-for-pg_log_query_plan.txt)
download | inline diff:
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
index b21a51aafe4..41951966838 100644
--- a/src/backend/commands/dynamic_explain.c
+++ b/src/backend/commands/dynamic_explain.c
@@ -22,6 +22,7 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/backend_status.h"
+#include "utils/injection_point.h"
/* Is plan node wrapping for query plan logging currently in progress? */
static bool WrapNodesInProgress = false;
@@ -80,6 +81,8 @@ LogQueryPlan(void)
ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+ INJECTION_POINT("logging-query-plan", NULL);
+
ereport(LOG_SERVER_ONLY,
errmsg("query and its plan running on backend with PID %d are:\n%s",
MyProcPid, es->str->data));
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index df5b55ae64d..db792e97c4b 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -59,6 +59,7 @@
#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/backend_status.h"
+#include "utils/injection_point.h"
#include "utils/lsyscache.h"
#include "utils/partcache.h"
#include "utils/rls.h"
@@ -333,6 +334,8 @@ standard_ExecutorRun(QueryDesc *queryDesc,
oldQueryDesc = GetCurrentQueryDesc();
SetCurrentQueryDesc(queryDesc);
+ INJECTION_POINT("standard-executor-run", NULL);
+
/*
* Switch into per-query memory context
*/
diff --git a/src/test/isolation/expected/pg_log_query_plan.out b/src/test/isolation/expected/pg_log_query_plan.out
new file mode 100644
index 00000000000..2d126785c18
--- /dev/null
+++ b/src/test/isolation/expected/pg_log_query_plan.out
@@ -0,0 +1,36 @@
+Parsed test spec with 2 sessions
+
+starting permutation: query1 logreq2 wakeup2 detach2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step query1: SELECT COUNT(*) FROM foo; <waiting ...>
+step logreq2: SELECT pg_log_query_plan(pid) FROM pg_stat_activity WHERE backend_type = 'client backend';
+pg_log_query_plan
+-----------------
+t
+t
+t
+(3 rows)
+
+step wakeup2: SELECT injection_points_wakeup('standard-executor-run');
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: notice triggered for injection point logging-query-plan
+step query1: <... completed>
+count
+-----
+ 100
+(1 row)
+
+step detach2: SELECT injection_points_detach('standard-executor-run');
+injection_points_detach
+-----------------------
+
+(1 row)
+
diff --git a/src/test/isolation/specs/pg_log_query_plan.spec b/src/test/isolation/specs/pg_log_query_plan.spec
new file mode 100644
index 00000000000..11c344d9fdd
--- /dev/null
+++ b/src/test/isolation/specs/pg_log_query_plan.spec
@@ -0,0 +1,27 @@
+# pg_log_query_plan() test
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE TABLE foo AS SELECT generate_series(1,100);
+}
+teardown
+{
+ DROP EXTENSION injection_points;
+ DROP TABLE foo;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('logging-query-plan', 'notice');
+ SELECT injection_points_attach('standard-executor-run', 'wait');
+}
+step query1 { SELECT COUNT(*) FROM foo; }
+
+session s2
+step logreq2 { SELECT pg_log_query_plan(pid) FROM pg_stat_activity WHERE backend_type = 'client backend'; }
+step wakeup2 { SELECT injection_points_wakeup('standard-executor-run'); }
+step detach2 { SELECT injection_points_detach('standard-executor-run'); }
+
+permutation query1 logreq2 wakeup2 detach2
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-10-16 20:15 ` Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2025-10-16 20:15 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On Wed, Oct 1, 2025 at 5:11 AM torikoshia <[email protected]> wrote:
> I was also considering using an isolation test and injection points,
> like in the attached PoC patch. The main steps are:
>
> In session1, set an injection point to wait during query execution.
> In session1, run a query that waits at the injection point.
> In session2, call pg_log_query_plan().
> In session2, wake up session1.
The key thing here is that we need to verify that each thing we do in
each session actually takes effect before doing the next thing. When
we're running an SQL statement to completion, nothing special is
needed: we just wait for completion -- but in any other case, we need
some kind of an explicit wait step after performing the action. Also,
I don't think we need the standard-executor-run injection point you've
introduced, because we have other ways to make query execution wait
already, such as (a) pg_sleep() with a very long timeout or (b)
pg_advisory_lock() on a lock held by another process. So I imagine the
outline maybe looking something like this:
S1: Set an injection point to wait in HandleLogQueryPlanInterrupt
[runs to completion].
S2: Take an advisory lock [runs to completion].
S1: Start a query that attempts to acquire the same advisory lock.
Use $node->wait_for_event() to be sure that S1 is now waiting on the lock.
S2: Commit, thus releasing the advisory lock [runs to completion].
Use $node->wait_for_event() to be sure that S1 is now waiting inside
HandleLogQueryPlanInterrupt.
Remember the log offset, as in src/test/modules/test_misc/t/005_timeouts.pl
Detach the injection point.
Use $node->wait_for_log() to wait for the expected log message to appear.
It's really hard to make these kinds of sequences race-free, so there
could well be bugs in the above outline, but I hope it is close to the
right thing.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2025-10-20 12:15 ` torikoshia <[email protected]>
2025-11-18 20:19 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: torikoshia @ 2025-10-20 12:15 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On 2025-10-17 05:15, Robert Haas wrote:
> On Wed, Oct 1, 2025 at 5:11 AM torikoshia <[email protected]>
> wrote:
>> I was also considering using an isolation test and injection points,
>> like in the attached PoC patch. The main steps are:
>>
>> In session1, set an injection point to wait during query execution.
>> In session1, run a query that waits at the injection point.
>> In session2, call pg_log_query_plan().
>> In session2, wake up session1.
>
> The key thing here is that we need to verify that each thing we do in
> each session actually takes effect before doing the next thing. When
> we're running an SQL statement to completion, nothing special is
> needed: we just wait for completion -- but in any other case, we need
> some kind of an explicit wait step after performing the action.
That makes sense.
> Also,
> I don't think we need the standard-executor-run injection point you've
> introduced, because we have other ways to make query execution wait
> already, such as (a) pg_sleep() with a very long timeout or (b)
> pg_advisory_lock() on a lock held by another process. So I imagine the
> outline maybe looking something like this:
>
> S1: Set an injection point to wait in HandleLogQueryPlanInterrupt
> [runs to completion].
> S2: Take an advisory lock [runs to completion].
> S1: Start a query that attempts to acquire the same advisory lock.
> Use $node->wait_for_event() to be sure that S1 is now waiting on the
> lock.
> S2: Commit, thus releasing the advisory lock [runs to completion].
> Use $node->wait_for_event() to be sure that S1 is now waiting inside
> HandleLogQueryPlanInterrupt.
> Remember the log offset, as in
> src/test/modules/test_misc/t/005_timeouts.pl
> Detach the injection point.
> Use $node->wait_for_log() to wait for the expected log message to
> appear.
>
> It's really hard to make these kinds of sequences race-free, so there
> could well be bugs in the above outline, but I hope it is close to the
> right thing.
Thanks for the detailed outline!
Attached patch adds a test following the suggestion.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
Attachments:
[text/x-diff] v50-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.4K, ../../[email protected]/2-v50-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 8fc81ac64ff0c3f9b77812f15eccc9a43a27b838 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 20 Oct 2025 21:02:54 +0900
Subject: [PATCH v50] Add function to log the plan of the currently running
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 24 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 34 ++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 187 ++++++++++++++++++
src/backend/commands/explain.c | 38 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 123 +++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 25 +++
src/include/commands/explain.h | 9 +-
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
.../test_misc/t/009_pg_log_query_plan.pl | 103 ++++++++++
src/test/regress/expected/misc_functions.out | 38 ++++
src/test/regress/sql/misc_functions.sql | 31 +++
24 files changed, 647 insertions(+), 34 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
create mode 100644 src/test/modules/test_misc/t/009_pg_log_query_plan.pl
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 1f4badb4928..6c4217c9d1f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -404,26 +405,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 1b465bc8ba7..66bfb5ab4df 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 2cf3d4e92b7..56c3c196f45 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -36,6 +36,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -215,6 +216,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -248,6 +250,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -933,6 +936,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2916,6 +2940,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current query plan state used for logging. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5308,6 +5335,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current query plan state used for logging. Note that even after
+ * this reset, it's still possible to obtain the parent transaction's
+ * query plans, since they are preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 2d946d6d9e9..67beafcc82c 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -782,6 +782,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index cb2fbdc7c60..5e83907eb17 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..95d4e330b2f
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,187 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+#include "utils/injection_point.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+#ifdef USE_INJECTION_POINTS
+ INJECTION_POINT("log-query-interrupt", NULL);
+#endif
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly and
+ * rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e6edae0845c..9f1e69d68d3 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1085,6 +1085,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1820,7 +1851,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1828,7 +1862,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index dd4cde41d32..4a64b8e9d09 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 713e926329c..663545e0e5b 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -312,6 +313,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -324,6 +326,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -386,6 +395,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeead..bf1e26b7af1 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -546,6 +564,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 087821311cc..07fcac7f0bc 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -691,6 +692,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7dd75a490aa..98b62a9e3a0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -36,6 +36,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/event_trigger.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
@@ -3539,6 +3540,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..927ccda0307 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index 4528e51829e..3c6e3b7f2ce 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern void CommandCounterIncrement(void);
extern void ForceSyncCommit(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b51d2b17379..8444bb075de 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8617,6 +8617,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..83a9e27deef
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,25 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 6e51d50efc7..0dc160fa7b4 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -70,8 +70,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
-extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
-extern void ExplainPrintTriggers(ExplainState *es,
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
extern void ExplainPrintJITSummary(ExplainState *es,
@@ -80,5 +82,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index ba073b86918..bf40fdabff5 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 3248e78cd28..c66df11e71a 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -295,6 +295,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..22b945e918c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index afeeb1ca019..ea26242c868 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/test/modules/test_misc/t/009_pg_log_query_plan.pl b/src/test/modules/test_misc/t/009_pg_log_query_plan.pl
new file mode 100644
index 00000000000..64dc78188a1
--- /dev/null
+++ b/src/test/modules/test_misc/t/009_pg_log_query_plan.pl
@@ -0,0 +1,103 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+use locale;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test that pg_log_query_plan() actually logs the query plan of
+# another backend executing a query.
+
+# This test requires timing cordinations:
+# 1) The target backend must be executing a query when
+# pg_log_query_plan() sends the signal.
+# 2) We must confirm that the target backend actually received the
+# signal that requests logging of the plan.
+#
+# We use an advisory lock and an injection point to control them
+# respectively.
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+my $psql_session1 = $node->background_psql('postgres');
+my $psql_session2 = $node->background_psql('postgres');
+
+my $session1_pid = $psql_session1->query_safe("select pg_backend_pid()");
+
+# Set injection point in the logging plan request handler to ensure
+# that session1 received the signal of pg_log_query_plan().
+$psql_session1->query_safe(
+ qq[
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('log-query-interrupt', 'wait');
+]);
+
+# Use an advisory lock to make session1 blocked during query execution.
+$psql_session2->query_safe(
+ qq[
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+]);
+
+$psql_session1->query_until(
+ qr/wait_on_advisory_lock/, q(
+ \echo wait_on_advisory_lock
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+));
+
+# Confirm that session1 is actually waiting on the advisory lock.
+$node->wait_for_event('client backend', 'advisory');
+
+# Run pg_log_query_plan().
+# Then commit the session 2 to release the advisory lock.
+$psql_session2->query_safe(
+ qq[
+ SELECT pg_log_query_plan($session1_pid);
+ COMMIT;
+]);
+
+# Ensure that the signal of pg_log_query_plan() is actually
+# rececived by confirming session1 is waiting on the injection point.
+$node->wait_for_event('client backend', 'log-query-interrupt');
+
+my $log_offset = -s $node->logfile;
+
+# Detach the injection point to start logging the plan.
+$psql_session2->query_safe(
+ qq[
+ SELECT injection_points_wakeup('log-query-interrupt');
+ SELECT injection_points_detach('log-query-interrupt');
+]);
+
+$node->wait_for_log('query and its plan running on backend with PID',
+ $log_offset);
+
+$psql_session1->query_safe("COMMIT;");
+
+ok($psql_session1->quit);
+ok($psql_session2->quit);
+
+done_testing();
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 36164a99c83..fc8c6fd0dc5 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -366,6 +366,44 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 23792c4132a..bd69ed550a8 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -143,6 +143,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
base-commit: 762faf702c6f7292bd02705553078700d92c15f1
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-11-18 20:19 ` Akshat Jaimini <[email protected]>
2025-11-19 03:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Akshat Jaimini @ 2025-11-18 20:19 UTC (permalink / raw)
To: [email protected]; +Cc: Atsushi Torikoshi <[email protected]>
Hi,
I have a question:
In src/backend/executor/execMain.c:
```
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
```
It would be really helpful if you could elaborate on any cases where this specific situation might arise i.e. where 'there was no time for logging the plan'. Are we referencing to something like a sudden shutdown of the postmaster process or is this referring to something else entirely?
Regards,
Akshat Jaimini
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-18 20:19 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
@ 2025-11-19 03:43 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2025-11-19 03:43 UTC (permalink / raw)
To: Akshat Jaimini <[email protected]>; +Cc: [email protected]
On 2025-11-19 05:19, Akshat Jaimini wrote:
Thanks for your review!
> Hi,
> I have a question:
>
> In src/backend/executor/execMain.c:
>
> ```
> + SetCurrentQueryDesc(oldQueryDesc);
> +
> + /*
> + * Ensure LogQueryPlanPending is initialized in case there was no
> time for
> + * logging the plan. Othewise plan will be logged at the next query
> + * execution on the same session.
> + */
> + LogQueryPlanPending = false;
> ```
>
> It would be really helpful if you could elaborate on any cases where
> this specific situation might arise i.e. where 'there was no time for
> logging the plan'. Are we referencing to something like a sudden
> shutdown of the postmaster process or is this referring to something
> else entirely?
What I have in mind are cases where a query finishes before
LogQueryPlan() is ever invoked.
Since LogQueryPlan() is called from ExecProcNodeFirst(), this generally
means pg_log_query_plan() was called at the moment just before query
execution completes.
Also, very short queries fall into this category:
=# select pg_log_query_plan(pg_backend_pid());
pg_log_query_plan
-------------------
t
(1 row)
=# select 1;
With the current patch, nothing is logged here.
But if I comment out the "LogQueryPlanPending = false" line, the plan
for "SELECT 1" ends up being logged:
LOG: 00000: query and its plan running on backend with PID 33040 are:
Query Text: select 1;
Result (cost=0.00..0.01 rows=1 width=4)
Output: 1
Settings: jit = 'off'
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-11-19 17:23 ` Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Robert Haas @ 2025-11-19 17:23 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On Mon, Oct 20, 2025 at 8:15 AM torikoshia <[email protected]> wrote:
> > S1: Set an injection point to wait in HandleLogQueryPlanInterrupt
> > [runs to completion].
> > S2: Take an advisory lock [runs to completion].
> > S1: Start a query that attempts to acquire the same advisory lock.
> > Use $node->wait_for_event() to be sure that S1 is now waiting on the
> > lock.
> > S2: Commit, thus releasing the advisory lock [runs to completion].
> > Use $node->wait_for_event() to be sure that S1 is now waiting inside
> > HandleLogQueryPlanInterrupt.
> > Remember the log offset, as in
> > src/test/modules/test_misc/t/005_timeouts.pl
> > Detach the injection point.
> > Use $node->wait_for_log() to wait for the expected log message to
> > appear.
> >
> > It's really hard to make these kinds of sequences race-free, so there
> > could well be bugs in the above outline, but I hope it is close to the
> > right thing.
>
> Thanks for the detailed outline!
> Attached patch adds a test following the suggestion.
Thanks. I'm not sure about this part:
+# Run pg_log_query_plan().
+# Then commit the session 2 to release the advisory lock.
+$psql_session2->query_safe(
+ qq[
+ SELECT pg_log_query_plan($session1_pid);
+ COMMIT;
+]);
This does two relevant things: one is to send a signal (the SELECT)
and the other is to release a lock (the COMMIT). Both of these events
will soon be perceived by the other session, but I am not sure whether
we have a guarantee about the order. If it's possible for the lock
release to be observed by the target session before the log-query-plan
interrupt arrives, the test will fail. If that is possible, I think we
should try to find a way to plug the gap. If it's not possible, I
think we should add a comment explaining why it can't happen.
Also, my apologies for taking some time to return to this thread.
Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2025-11-20 01:52 ` torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-11-20 01:52 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On 2025-11-20 02:23, Robert Haas wrote:
> On Mon, Oct 20, 2025 at 8:15 AM torikoshia <[email protected]>
> wrote:
>> > S1: Set an injection point to wait in HandleLogQueryPlanInterrupt
>> > [runs to completion].
>> > S2: Take an advisory lock [runs to completion].
>> > S1: Start a query that attempts to acquire the same advisory lock.
>> > Use $node->wait_for_event() to be sure that S1 is now waiting on the
>> > lock.
>> > S2: Commit, thus releasing the advisory lock [runs to completion].
>> > Use $node->wait_for_event() to be sure that S1 is now waiting inside
>> > HandleLogQueryPlanInterrupt.
>> > Remember the log offset, as in
>> > src/test/modules/test_misc/t/005_timeouts.pl
>> > Detach the injection point.
>> > Use $node->wait_for_log() to wait for the expected log message to
>> > appear.
>> >
>> > It's really hard to make these kinds of sequences race-free, so there
>> > could well be bugs in the above outline, but I hope it is close to the
>> > right thing.
>>
>> Thanks for the detailed outline!
>> Attached patch adds a test following the suggestion.
>
> Thanks. I'm not sure about this part:
>
> +# Run pg_log_query_plan().
> +# Then commit the session 2 to release the advisory lock.
> +$psql_session2->query_safe(
> + qq[
> + SELECT pg_log_query_plan($session1_pid);
> + COMMIT;
> +]);
>
> This does two relevant things: one is to send a signal (the SELECT)
> and the other is to release a lock (the COMMIT). Both of these events
> will soon be perceived by the other session, but I am not sure whether
> we have a guarantee about the order. If it's possible for the lock
> release to be observed by the target session before the log-query-plan
> interrupt arrives, the test will fail. If that is possible, I think we
> should try to find a way to plug the gap.
I think it's possible.
Updated the test so that COMMIT happens only after we confirm that the
backend is already waiting at the injection point.
> Also, my apologies for taking some time to return to this thread.
Not at all -- I really appreciate your continued review.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
Attachments:
[text/x-diff] v51-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.4K, ../../[email protected]/2-v51-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 019829ef95d9b145cea530c4bf07f8ea3ef61c88 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Thu, 20 Nov 2025 10:33:05 +0900
Subject: [PATCH v51] Add function to log the plan of the currently running
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 24 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 34 ++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 187 ++++++++++++++++++
src/backend/commands/explain.c | 38 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 123 +++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/dynamic_explain.h | 25 +++
src/include/commands/explain.h | 9 +-
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
.../test_misc/t/010_pg_log_query_plan.pl | 101 ++++++++++
src/test/regress/expected/misc_functions.out | 38 ++++
src/test/regress/sql/misc_functions.sql | 31 +++
24 files changed, 645 insertions(+), 34 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
create mode 100644 src/test/modules/test_misc/t/010_pg_log_query_plan.pl
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 1f4badb4928..6c4217c9d1f 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -15,6 +15,7 @@
#include <limits.h>
#include "access/parallel.h"
+#include "commands/dynamic_explain.h"
#include "commands/explain.h"
#include "commands/explain_format.h"
#include "commands/explain_state.h"
@@ -404,26 +405,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 1b465bc8ba7..66bfb5ab4df 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 092e197eba3..be5a9cef8a9 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -37,6 +37,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -216,6 +217,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -249,6 +251,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -934,6 +937,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2922,6 +2946,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current query plan state used for logging. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5314,6 +5341,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current query plan state used for logging. Note that even after
+ * this reset, it's still possible to obtain the parent transaction's
+ * query plans, since they are preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 2d946d6d9e9..67beafcc82c 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -782,6 +782,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index f99acfd2b4b..ac8e3996ae9 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -32,6 +32,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..95d4e330b2f
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,187 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "utils/backend_status.h"
+#include "utils/injection_point.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+#ifdef USE_INJECTION_POINTS
+ INJECTION_POINT("log-query-interrupt", NULL);
+#endif
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly and
+ * rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 7e699f8595e..01d343f50b8 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1085,6 +1085,37 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1820,7 +1851,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1828,7 +1862,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index 9f640ad4810..69010ad5ec9 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -20,6 +20,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 27c9eec697b..8ce81aace52 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -43,6 +43,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -312,6 +313,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -324,6 +326,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -386,6 +395,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStopNode(queryDesc->totaltime, estate->es_processed);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f5f9cfbeead..bf1e26b7af1 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/nodeAgg.h"
#include "executor/nodeAppend.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -546,6 +564,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 087821311cc..07fcac7f0bc 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -691,6 +692,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7dd75a490aa..98b62a9e3a0 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -36,6 +36,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/event_trigger.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
@@ -3539,6 +3540,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..927ccda0307 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index 4528e51829e..3c6e3b7f2ce 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern void CommandCounterIncrement(void);
extern void ForceSyncCommit(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index aaadfd8c748..a930a673610 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8617,6 +8617,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..83a9e27deef
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,25 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 6e51d50efc7..0dc160fa7b4 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -70,8 +70,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
-extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
-extern void ExplainPrintTriggers(ExplainState *es,
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
extern void ExplainPrintJITSummary(ExplainState *es,
@@ -80,5 +82,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index ba073b86918..bf40fdabff5 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -71,6 +71,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index fa2b657fb2f..bf2822d6e9b 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -298,6 +298,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 9a7d733ddef..772c01c95e1 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index afeeb1ca019..ea26242c868 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/test/modules/test_misc/t/010_pg_log_query_plan.pl b/src/test/modules/test_misc/t/010_pg_log_query_plan.pl
new file mode 100644
index 00000000000..817ff3a1dcd
--- /dev/null
+++ b/src/test/modules/test_misc/t/010_pg_log_query_plan.pl
@@ -0,0 +1,101 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+use locale;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test that pg_log_query_plan() actually logs the query plan of
+# another backend executing a query.
+
+# This test requires timing cordinations:
+# 1) The target backend must be executing a query when
+# pg_log_query_plan() sends the signal.
+# 2) We must confirm that the target backend actually received the
+# signal that requests logging of the plan.
+#
+# We use an advisory lock and an injection point to control them
+# respectively.
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+my $psql_session1 = $node->background_psql('postgres');
+my $psql_session2 = $node->background_psql('postgres');
+
+my $session1_pid = $psql_session1->query_safe("select pg_backend_pid()");
+
+# Set injection point in the logging plan request handler to ensure
+# that session1 received the signal of pg_log_query_plan().
+$psql_session1->query_safe(
+ qq[
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('log-query-interrupt', 'wait');
+]);
+
+# Use an advisory lock to make session1 blocked during query execution.
+$psql_session2->query_safe(
+ qq[
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+]);
+
+$psql_session1->query_until(
+ qr/wait_on_advisory_lock/, q(
+ \echo wait_on_advisory_lock
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+));
+
+# Confirm that session1 is actually waiting on the advisory lock.
+$node->wait_for_event('client backend', 'advisory');
+
+# Run pg_log_query_plan().
+$psql_session2->query_safe("SELECT pg_log_query_plan($session1_pid);");
+
+# Ensure that the signal of pg_log_query_plan() is actually
+# rececived by confirming session1 is waiting on the injection point.
+$node->wait_for_event('client backend', 'log-query-interrupt');
+
+# Commit the session 2 to release the advisory lock.
+$psql_session2->query_safe("COMMIT;");
+
+my $log_offset = -s $node->logfile;
+
+# Detach the injection point to start logging the plan.
+$psql_session2->query_safe(
+ qq[
+ SELECT injection_points_wakeup('log-query-interrupt');
+ SELECT injection_points_detach('log-query-interrupt');
+]);
+
+$node->wait_for_log('query and its plan running on backend with PID',
+ $log_offset);
+
+$psql_session1->query_safe("COMMIT;");
+
+ok($psql_session1->quit);
+ok($psql_session2->quit);
+
+done_testing();
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index e76e28b95ce..b53b1f784f6 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -397,6 +397,44 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 220472d5ad1..4b068c71220 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -154,6 +154,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
base-commit: aaf035790aebb4de6d85b60f8f3089c3c656b325
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2025-11-20 13:17 ` Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2025-11-20 13:17 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On Wed, Nov 19, 2025 at 8:52 PM torikoshia <[email protected]> wrote:
> I think it's possible.
> Updated the test so that COMMIT happens only after we confirm that the
> backend is already waiting at the injection point.
Thanks, that looks better to me. Looking at this version, I wondered
whether this was OK:
+# Commit the session 2 to release the advisory lock.
+$psql_session2->query_safe("COMMIT;");
+
+my $log_offset = -s $node->logfile;
If the COMMIT causes something to appear in the log, we're not
guaranteed as to whether that will happen before or after we record
the log offset. However, the string for which we're looking will (if I
understand correctly) only appear after the wakeup/detach from the
injection point, so I don't think there's any real problem here.
A couple of testing suggestions, if you haven't already:
1. Run the test in a loop, say 100 times, to check for random failures.
2. Insert a sleep(10) after each line of the strict in turn and run it
(perhaps a few tiimes) and check to make sure that the tests still
pass. (I don't mean put sleep(10) after every line -- I mean put a
sleep(10) in one place at a time and keep moving it after you've
verified that the current location doesn't cause a failure.)
I know from experience that it's quite hard to get these tests to be
fully reliable and I won't feel too bad if it turns out we missed
something, but at least we can try.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2025-11-25 12:43 ` torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2025-11-25 12:43 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On 2025-11-20 22:17, Robert Haas wrote:
> On Wed, Nov 19, 2025 at 8:52 PM torikoshia <[email protected]>
> wrote:
>> I think it's possible.
>> Updated the test so that COMMIT happens only after we confirm that the
>> backend is already waiting at the injection point.
>
> Thanks, that looks better to me. Looking at this version, I wondered
> whether this was OK:
>
> +# Commit the session 2 to release the advisory lock.
> +$psql_session2->query_safe("COMMIT;");
> +
> +my $log_offset = -s $node->logfile;
>
> If the COMMIT causes something to appear in the log, we're not
> guaranteed as to whether that will happen before or after we record
> the log offset. However, the string for which we're looking will (if I
> understand correctly) only appear after the wakeup/detach from the
> injection point, so I don't think there's any real problem here.
I have the same understanding.
> A couple of testing suggestions, if you haven't already:
Thank you for the suggestions.
> 1. Run the test in a loop, say 100 times, to check for random failures.
I ran 010_pg_log_query_plan.pl 300 times, but it never failed.
> 2. Insert a sleep(10) after each line of the strict in turn and run it
> (perhaps a few tiimes) and check to make sure that the tests still
> pass. (I don't mean put sleep(10) after every line -- I mean put a
> sleep(10) in one place at a time and keep moving it after you've
> verified that the current location doesn't cause a failure.)
I inserted sleep(10) at each line in turn and ran the test 10 times for
each location, but fortunately (or unfortunately?) I did not observe any
failures.
> I know from experience that it's quite hard to get these tests to be
> fully reliable and I won't feel too bad if it turns out we missed
> something, but at least we can try.
BTW to test not 010_pg_log_query_plan but pg_log_query_plan(), I also
invoked pg_log_query_plan() in 3 parallel sessions at 0.1-second
intervals(\watch 0.1) during running `make intallcheck` 10 times, and I
did not see any crashes.
--
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA Japan Corporation to SRA OSS K.K.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2026-06-08 06:08 ` Lukas Fittl <[email protected]>
2026-06-08 11:48 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: Lukas Fittl @ 2026-06-08 06:08 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
Hi,
On Mon, Mar 16, 2026 at 8:14 AM torikoshia <[email protected]> wrote:
>
> Rebased the patch.
I was reminded about this patch at PG DATA in Chicago last week, and
figured I'd see if I can contribute a review / help move this forward
in PG20.
First of all, find attached a v54 that is rebased over changes late in
the PG19 cycle, with other minimal fixes:
- Add a missing call to explain_per_plan_hook in ExplainStringAssemble
(this hook was newly added after your last patch version)
- The TAP test number (11) was overlapping with an existing test
(011_lock_stats) that was recently added, renumbered to
014_pg_log_query_plan.pl
- Add missing reference for the TAP test in Meson builds
- Adjust auto_explain.c to not include "dynamic_explain.h" (its not
needed, ExplainStringAssemble is defined in explain.h which is already
included)
- pgindent run to fix minor whitespace issues in dynamic_explain.c
I've also read through the other thread re: progressive explain, and
it seems like this infrastructure here to log the current plan
(without ANALYZE) would be required in any case (and I think the
overall approach makes sense), and then separately we can solve the
progressive explain's parallel worker and other design issues around
execution statistics.
In terms of code review:
> diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
> index aafc53e0164..09e51b3d75a 100644
> --- a/src/backend/access/transam/xact.c
> +++ b/src/backend/access/transam/xact.c
> @@ -37,6 +37,7 @@
> #include "catalog/pg_enum.h"
> #include "catalog/storage.h"
> #include "commands/async.h"
> +#include "commands/dynamic_explain.h"
> #include "commands/tablecmds.h"
> #include "commands/trigger.h"
> #include "common/pg_prng.h"
> @@ -217,6 +218,7 @@ typedef struct TransactionStateData
> bool parallelChildXact; /* is any parent transaction parallel? */
> bool chain; /* start a new block after this one */
> bool topXidLogged; /* for a subxact: is top-level XID logged? */
> + QueryDesc *queryDesc; /* my current QueryDesc */
> struct TransactionStateData *parent; /* back link to parent */
> } TransactionStateData;
Somehow putting the current QueryDesc into TransactionStateData seemed
a bit odd to me, and I briefly experimented with putting this into the
active portal instead - but that doesn't work with subtransactions
that don't have their own portal, which presumably is how you landed
on this design. So I think this makes sense as-is, given those
challenges.
> @@ -2953,6 +2977,9 @@ AbortTransaction(void)
> /* Reset snapshot export state. */
> SnapBuildResetExportedSnapshotState();
>
> + /* Reset current query plan state used for logging. */
> + SetCurrentQueryDesc(NULL);
> +
> /*
> * If this xact has started any unfinished parallel operation, clean up
> * its workers and exit parallel mode. Don't warn about leaked resources.
> @@ -5352,6 +5379,13 @@ AbortSubTransaction(void)
> /* Reset logical streaming state. */
> ResetLogicalStreamingState();
>
> + /*
> + * Reset current query plan state used for logging. Note that even after
> + * this reset, it's still possible to obtain the parent transaction's
> + * query plans, since they are preserved in standard_ExecutorRun().
> + */
> + SetCurrentQueryDesc(NULL);
> +
> /*
> * No need for SnapBuildResetExportedSnapshotState() here, snapshot
> * exports are not supported in subtransactions.
It seems better to me if we avoided overly specific code comments in
xact.c that talk about plan logging behavior, i.e. look at this more
as a general purpose "what's the current queryDesc for the current
transaction" facility.
> diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
> index 7e40b852517..96cb22daf80 100644
> --- a/src/backend/executor/execProcnode.c
> +++ b/src/backend/executor/execProcnode.c
> @@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
> ...
> static TupleTableSlot *
> ExecProcNodeFirst(PlanState *node)
> {
> ...
> + if (LogQueryPlanPending)
> + LogQueryPlan();
> +
> /*
> * If instrumentation is required, change the wrapper to one that just
> * does instrumentation. Otherwise we can dispense with all wrappers and
> * have ExecProcNode() directly call the relevant function from now on.
> */
> - if (node->instrument)
> + else if (node->instrument)
> node->ExecProcNode = ExecProcNodeInstr;
> else
> node->ExecProcNode = node->ExecProcNodeReal;
I'm not sure if this was discussed earlier already, but I find the
flow here a bit hard to follow in regards to how we execute the actual
node function again.
Specifically, since this uses "else if" we don't touch ExecProcNode
when LogQueryPlanPending = true, and since node->ExecProcNode still
points to ExecProcNodeFirst, the function basically calls itself. On
the second go around we assume that LogQueryPlanPending is no longer
set, so we get back to the intended ExecProcNode *if*
LogQueryPlanPending was set to false (which it is in LogQueryPlan, but
that's not easy to see from just that function).
Why not do it like this:
if (LogQueryPlanPending)
LogQueryPlan();
/* Unmodified existing code */
if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
(it seems like the patch suggestion that Robert had earlier did this,
but not clear why it evolved from that?)
Thanks,
Lukas
--
Lukas Fittl
Attachments:
[application/octet-stream] v54-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.7K, ../../CAP53Pkw8CX0RQY474McHyc1N8p331-asEBAEGu0X+bJX-VB8jg@mail.gmail.com/2-v54-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From cd3d7a32c55eccfd27ee4395a7497969b0b2c3cc Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 16 Mar 2026 23:58:06 +0900
Subject: [PATCH v54] Add function to log the plan of the currently running
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 29 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 34 ++++
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 188 ++++++++++++++++++
src/backend/commands/explain.c | 44 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 123 +++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 7 +
src/include/commands/dynamic_explain.h | 25 +++
src/include/commands/explain.h | 9 +-
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/modules/test_misc/meson.build | 1 +
.../test_misc/t/014_pg_log_query_plan.pl | 101 ++++++++++
src/test/regress/expected/misc_functions.out | 38 ++++
src/test/regress/sql/misc_functions.sql | 31 +++
24 files changed, 651 insertions(+), 40 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
create mode 100644 src/test/modules/test_misc/t/014_pg_log_query_plan.pl
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 97ce498d0c1..07014cab366 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -451,32 +451,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
apply_extension_options(es, extension_options);
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- if (explain_per_plan_hook)
- (*explain_per_plan_hook) (queryDesc->plannedstmt,
- NULL, es,
- queryDesc->sourceText,
- queryDesc->params,
- queryDesc->estate->es_queryEnv);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 0eae1c1f616..ba319c94160 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 5586fbe5b07..d6031f97ca7 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -37,6 +37,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -217,6 +218,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -250,6 +252,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -935,6 +938,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2953,6 +2977,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current query plan state used for logging. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5352,6 +5379,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current query plan state used for logging. Note that even after
+ * this reset, it's still possible to obtain the parent transaction's
+ * query plans, since they are preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 5b9d084977e..8f13c7f5752 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -31,6 +31,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..4f0c79f2638
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,188 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "storage/procsignal.h"
+#include "utils/backend_status.h"
+#include "utils/injection_point.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+#ifdef USE_INJECTION_POINTS
+ INJECTION_POINT("log-query-interrupt", NULL);
+#endif
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly
+ * and rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 112c17b0d64..51648b5d510 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1092,6 +1092,43 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ if (explain_per_plan_hook)
+ (*explain_per_plan_hook) (queryDesc->plannedstmt,
+ NULL, es,
+ queryDesc->sourceText,
+ queryDesc->params,
+ queryDesc->estate->es_queryEnv);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1827,7 +1864,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1875,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index 9f258d566eb..2f2b30cea0c 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -19,6 +19,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4b30f768680..c749cae8008 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -44,6 +44,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -323,6 +324,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -335,6 +337,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -397,6 +406,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStop(queryDesc->query_instr);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Othewise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 7c4c66e323f..25e5541eecd 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeAgg.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
* have ExecProcNode() directly call the relevant function from now on.
*/
- if (node->instrument)
+ else if (node->instrument)
node->ExecProcNode = ExecProcNodeInstr;
else
node->ExecProcNode = node->ExecProcNodeReal;
@@ -527,6 +545,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 1397f65f67b..bb01f605d00 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/repack.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -713,6 +714,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index dbef734a93f..98413ad58fd 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -36,6 +36,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/event_trigger.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
@@ -3609,6 +3610,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index bbd28d14d99..de84fd9cfa3 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index a8cbdf247c8..f2881d956c0 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern int GetTopReadOnlyTransactionNestLevel(void);
extern void CommandCounterIncrement(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be157a5fbe9..92976b1e532 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8721,6 +8721,13 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts', proacl => '{POSTGRES=X}' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan',
+ proacl => '{POSTGRES=X}' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..83a9e27deef
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,25 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 472e141bba3..0186e01df1c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -71,8 +71,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
-extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
-extern void ExplainPrintTriggers(ExplainState *es,
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintPlan(struct ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(struct ExplainState *es,
QueryDesc *queryDesc);
extern void ExplainPrintJITSummary(ExplainState *es,
@@ -81,5 +83,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(struct ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 97bc7ed49f6..b9a98157815 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -73,6 +73,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 33bbdbfeffb..777922d3846 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -299,6 +299,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 7de0a115402..2374a0079df 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -98,6 +98,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index aaa158bfd66..c2142043e2d 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
PROCSIG_SLOTSYNC_MESSAGE, /* ask slot synchronization to stop */
PROCSIG_REPACK_MESSAGE, /* Message from repack worker */
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index 969e90b396d..4c23bb794c7 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -22,6 +22,7 @@ tests += {
't/011_lock_stats.pl',
't/012_ddlutils.pl',
't/013_temp_obj_multisession.pl',
+ 't/014_pg_log_query_plan.pl',
],
# The injection points are cluster-wide, so disable installcheck
'runningcheck': false,
diff --git a/src/test/modules/test_misc/t/014_pg_log_query_plan.pl b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
new file mode 100644
index 00000000000..817ff3a1dcd
--- /dev/null
+++ b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
@@ -0,0 +1,101 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+use locale;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test that pg_log_query_plan() actually logs the query plan of
+# another backend executing a query.
+
+# This test requires timing cordinations:
+# 1) The target backend must be executing a query when
+# pg_log_query_plan() sends the signal.
+# 2) We must confirm that the target backend actually received the
+# signal that requests logging of the plan.
+#
+# We use an advisory lock and an injection point to control them
+# respectively.
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+my $psql_session1 = $node->background_psql('postgres');
+my $psql_session2 = $node->background_psql('postgres');
+
+my $session1_pid = $psql_session1->query_safe("select pg_backend_pid()");
+
+# Set injection point in the logging plan request handler to ensure
+# that session1 received the signal of pg_log_query_plan().
+$psql_session1->query_safe(
+ qq[
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('log-query-interrupt', 'wait');
+]);
+
+# Use an advisory lock to make session1 blocked during query execution.
+$psql_session2->query_safe(
+ qq[
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+]);
+
+$psql_session1->query_until(
+ qr/wait_on_advisory_lock/, q(
+ \echo wait_on_advisory_lock
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+));
+
+# Confirm that session1 is actually waiting on the advisory lock.
+$node->wait_for_event('client backend', 'advisory');
+
+# Run pg_log_query_plan().
+$psql_session2->query_safe("SELECT pg_log_query_plan($session1_pid);");
+
+# Ensure that the signal of pg_log_query_plan() is actually
+# rececived by confirming session1 is waiting on the injection point.
+$node->wait_for_event('client backend', 'log-query-interrupt');
+
+# Commit the session 2 to release the advisory lock.
+$psql_session2->query_safe("COMMIT;");
+
+my $log_offset = -s $node->logfile;
+
+# Detach the injection point to start logging the plan.
+$psql_session2->query_safe(
+ qq[
+ SELECT injection_points_wakeup('log-query-interrupt');
+ SELECT injection_points_detach('log-query-interrupt');
+]);
+
+$node->wait_for_log('query and its plan running on backend with PID',
+ $log_offset);
+
+$psql_session1->query_safe("COMMIT;");
+
+ok($psql_session1->quit);
+ok($psql_session2->quit);
+
+done_testing();
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3261bff209..5d310939a75 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -357,6 +357,44 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 946ee5726cd..e7ab81cca60 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -113,6 +113,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.47.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
@ 2026-06-08 11:48 ` Atsushi Torikoshi <[email protected]>
1 sibling, 0 replies; 124+ messages in thread
From: Atsushi Torikoshi @ 2026-06-08 11:48 UTC (permalink / raw)
To: Lukas Fittl <[email protected]>; +Cc: torikoshia <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
On Mon, Jun 8, 2026 at 3:08 PM Lukas Fittl <[email protected]> wrote:
> I was reminded about this patch at PG DATA in Chicago last week, and
> figured I'd see if I can contribute a review / help move this forward
> in PG20.
Thanks for your help! I really appreciate it.
> First of all, find attached a v54 that is rebased over changes late in
> the PG19 cycle, with other minimal fixes:
It may take me a few days, but I'll take a look at the updated patch.
--
Atsushi Torikoshi
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
@ 2026-06-15 13:26 ` torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: torikoshia @ 2026-06-15 13:26 UTC (permalink / raw)
To: Lukas Fittl <[email protected]>; +Cc: [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]; [email protected]
On 2026-06-08 15:08, Lukas Fittl wrote:
Thanks for your review and update again.
> Hi,
>
> On Mon, Mar 16, 2026 at 8:14 AM torikoshia <[email protected]>
> wrote:
>>
>> Rebased the patch.
>
> I was reminded about this patch at PG DATA in Chicago last week, and
> figured I'd see if I can contribute a review / help move this forward
> in PG20.
>
> First of all, find attached a v54 that is rebased over changes late in
> the PG19 cycle, with other minimal fixes:
>
> - Add a missing call to explain_per_plan_hook in ExplainStringAssemble
> (this hook was newly added after your last patch version)
> - The TAP test number (11) was overlapping with an existing test
> (011_lock_stats) that was recently added, renumbered to
> 014_pg_log_query_plan.pl
> - Add missing reference for the TAP test in Meson builds
> - Adjust auto_explain.c to not include "dynamic_explain.h" (its not
> needed, ExplainStringAssemble is defined in explain.h which is already
> included)
> - pgindent run to fix minor whitespace issues in dynamic_explain.c
Agreed on those changes.
> I've also read through the other thread re: progressive explain, and
> it seems like this infrastructure here to log the current plan
> (without ANALYZE) would be required in any case (and I think the
> overall approach makes sense), and then separately we can solve the
> progressive explain's parallel worker and other design issues around
> execution statistics.
>
> In terms of code review:
>
>> diff --git a/src/backend/access/transam/xact.c
>> b/src/backend/access/transam/xact.c
>> index aafc53e0164..09e51b3d75a 100644
>> --- a/src/backend/access/transam/xact.c
>> +++ b/src/backend/access/transam/xact.c
>> @@ -37,6 +37,7 @@
>> #include "catalog/pg_enum.h"
>> #include "catalog/storage.h"
>> #include "commands/async.h"
>> +#include "commands/dynamic_explain.h"
>> #include "commands/tablecmds.h"
>> #include "commands/trigger.h"
>> #include "common/pg_prng.h"
>> @@ -217,6 +218,7 @@ typedef struct TransactionStateData
>> bool parallelChildXact; /* is any parent transaction
>> parallel? */
>> bool chain; /* start a new block after this one
>> */
>> bool topXidLogged; /* for a subxact: is top-level XID
>> logged? */
>> + QueryDesc *queryDesc; /* my current QueryDesc */
>> struct TransactionStateData *parent; /* back link to parent */
>> } TransactionStateData;
>
> Somehow putting the current QueryDesc into TransactionStateData seemed
> a bit odd to me, and I briefly experimented with putting this into the
> active portal instead - but that doesn't work with subtransactions
> that don't have their own portal, which presumably is how you landed
> on this design. So I think this makes sense as-is, given those
> challenges.
>
>> @@ -2953,6 +2977,9 @@ AbortTransaction(void)
>> /* Reset snapshot export state. */
>> SnapBuildResetExportedSnapshotState();
>>
>> + /* Reset current query plan state used for logging. */
>> + SetCurrentQueryDesc(NULL);
>> +
>> /*
>> * If this xact has started any unfinished parallel operation,
>> clean up
>> * its workers and exit parallel mode. Don't warn about leaked
>> resources.
>> @@ -5352,6 +5379,13 @@ AbortSubTransaction(void)
>> /* Reset logical streaming state. */
>> ResetLogicalStreamingState();
>>
>> + /*
>> + * Reset current query plan state used for logging. Note that
>> even after
>> + * this reset, it's still possible to obtain the parent
>> transaction's
>> + * query plans, since they are preserved in
>> standard_ExecutorRun().
>> + */
>> + SetCurrentQueryDesc(NULL);
>> +
>> /*
>> * No need for SnapBuildResetExportedSnapshotState() here,
>> snapshot
>> * exports are not supported in subtransactions.
>
> It seems better to me if we avoided overly specific code comments in
> xact.c that talk about plan logging behavior, i.e. look at this more
> as a general purpose "what's the current queryDesc for the current
> transaction" facility.
Removed specific comments about plan logging.
>> diff --git a/src/backend/executor/execProcnode.c
>> b/src/backend/executor/execProcnode.c
>> index 7e40b852517..96cb22daf80 100644
>> --- a/src/backend/executor/execProcnode.c
>> +++ b/src/backend/executor/execProcnode.c
>> @@ -441,27 +443,43 @@ ExecSetExecProcNode(PlanState *node,
>> ExecProcNodeMtd function)
>> ...
>> static TupleTableSlot *
>> ExecProcNodeFirst(PlanState *node)
>> {
>> ...
>> + if (LogQueryPlanPending)
>> + LogQueryPlan();
>> +
>> /*
>> * If instrumentation is required, change the wrapper to one that
>> just
>> * does instrumentation. Otherwise we can dispense with all
>> wrappers and
>> * have ExecProcNode() directly call the relevant function from
>> now on.
>> */
>> - if (node->instrument)
>> + else if (node->instrument)
>> node->ExecProcNode = ExecProcNodeInstr;
>> else
>> node->ExecProcNode = node->ExecProcNodeReal;
>
> I'm not sure if this was discussed earlier already, but I find the
> flow here a bit hard to follow in regards to how we execute the actual
> node function again.
>
> Specifically, since this uses "else if" we don't touch ExecProcNode
> when LogQueryPlanPending = true, and since node->ExecProcNode still
> points to ExecProcNodeFirst, the function basically calls itself. On
> the second go around we assume that LogQueryPlanPending is no longer
> set, so we get back to the intended ExecProcNode *if*
> LogQueryPlanPending was set to false (which it is in LogQueryPlan, but
> that's not easy to see from just that function).
>
> Why not do it like this:
>
> if (LogQueryPlanPending)
> LogQueryPlan();
>
> /* Unmodified existing code */
> if (node->instrument)
> node->ExecProcNode = ExecProcNodeInstr;
> else
> node->ExecProcNode = node->ExecProcNodeReal;
>
Make sense. Modified it.
I also fixed some typos in the attached patch.
Thanks,
--
Atsushi Torikoshi
Seconded from NTT DATA CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v55-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.2K, ../../[email protected]/2-v55-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 6698abfe569bcd6ba5e7998001cd10e9d13b4baf Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 15 Jun 2026 22:18:24 +0900
Subject: [PATCH v55] Add function to log the plan of the currently running
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 29 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 34 ++++
src/backend/commands/Makefile | 1 +
src/backend/commands/dynamic_explain.c | 188 ++++++++++++++++++
src/backend/commands/explain.c | 44 +++-
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 121 ++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 7 +
src/include/commands/dynamic_explain.h | 25 +++
src/include/commands/explain.h | 5 +
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/modules/test_misc/meson.build | 1 +
.../test_misc/t/014_pg_log_query_plan.pl | 101 ++++++++++
src/test/regress/expected/misc_functions.out | 38 ++++
src/test/regress/sql/misc_functions.sql | 31 +++
24 files changed, 648 insertions(+), 37 deletions(-)
create mode 100644 src/backend/commands/dynamic_explain.c
create mode 100644 src/include/commands/dynamic_explain.h
create mode 100644 src/test/modules/test_misc/t/014_pg_log_query_plan.pl
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 97ce498d0c1..07014cab366 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -451,32 +451,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
apply_extension_options(es, extension_options);
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- if (explain_per_plan_hook)
- (*explain_per_plan_hook) (queryDesc->plannedstmt,
- NULL, es,
- queryDesc->sourceText,
- queryDesc->params,
- queryDesc->estate->es_queryEnv);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 0eae1c1f616..ba319c94160 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 5586fbe5b07..a2d880befe6 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -37,6 +37,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -217,6 +218,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -250,6 +252,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -935,6 +938,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2953,6 +2977,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current query plan state. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5352,6 +5379,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current query plan state. Note that even after this reset, it's
+ * still possible to obtain the parent transaction's query plans, since
+ * they are preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 5b9d084977e..8f13c7f5752 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -31,6 +31,7 @@ OBJS = \
define.o \
discard.o \
dropcmds.o \
+ dynamic_explain.o \
event_trigger.o \
explain.o \
explain_dr.o \
diff --git a/src/backend/commands/dynamic_explain.c b/src/backend/commands/dynamic_explain.c
new file mode 100644
index 00000000000..0f63e6a3a81
--- /dev/null
+++ b/src/backend/commands/dynamic_explain.c
@@ -0,0 +1,188 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/dynamic_explain.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/dynamic_explain.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "storage/procsignal.h"
+#include "utils/backend_status.h"
+#include "utils/injection_point.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+#ifdef USE_INJECTION_POINTS
+ INJECTION_POINT("log-query-interrupt", NULL);
+#endif
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly
+ * and rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 112c17b0d64..51648b5d510 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1092,6 +1092,43 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ if (explain_per_plan_hook)
+ (*explain_per_plan_hook) (queryDesc->plannedstmt,
+ NULL, es,
+ queryDesc->sourceText,
+ queryDesc->params,
+ queryDesc->estate->es_queryEnv);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1827,7 +1864,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1875,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index 9f258d566eb..2f2b30cea0c 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -19,6 +19,7 @@ backend_sources += files(
'define.c',
'discard.c',
'dropcmds.c',
+ 'dynamic_explain.c',
'event_trigger.c',
'explain.c',
'explain_dr.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4b30f768680..4b9f4f2b7a8 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -44,6 +44,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/dynamic_explain.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -323,6 +324,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -335,6 +337,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -397,6 +406,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStop(queryDesc->query_instr);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Otherwise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 7c4c66e323f..081597a31d0 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/dynamic_explain.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeAgg.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,21 +443,37 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
@@ -527,6 +545,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 1397f65f67b..bb01f605d00 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/repack.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -713,6 +714,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index dbef734a93f..98413ad58fd 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -36,6 +36,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/dynamic_explain.h"
#include "commands/event_trigger.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
@@ -3609,6 +3610,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index bbd28d14d99..de84fd9cfa3 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index a8cbdf247c8..f2881d956c0 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern int GetTopReadOnlyTransactionNestLevel(void);
extern void CommandCounterIncrement(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be157a5fbe9..92976b1e532 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8721,6 +8721,13 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts', proacl => '{POSTGRES=X}' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan',
+ proacl => '{POSTGRES=X}' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/dynamic_explain.h b/src/include/commands/dynamic_explain.h
new file mode 100644
index 00000000000..f3c4f6fb5af
--- /dev/null
+++ b/src/include/commands/dynamic_explain.h
@@ -0,0 +1,25 @@
+/*-------------------------------------------------------------------------
+ *
+ * dynamic_explain.h
+ * prototypes for dynamic_explain.c
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/dynamic_explain.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DYNAMIC_EXPLAIN_H
+#define DYNAMIC_EXPLAIN_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* DYNAMIC_EXPLAIN_H */
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 472e141bba3..42f1f7d61ff 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -71,6 +71,8 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es,
QueryDesc *queryDesc);
@@ -81,5 +83,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 97bc7ed49f6..b9a98157815 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -73,6 +73,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 650baab3efc..12a1673d00c 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -299,6 +299,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 7170a4bff98..0ab775e964f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -98,6 +98,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index aaa158bfd66..c2142043e2d 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
PROCSIG_SLOTSYNC_MESSAGE, /* ask slot synchronization to stop */
PROCSIG_REPACK_MESSAGE, /* Message from repack worker */
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index 969e90b396d..4c23bb794c7 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -22,6 +22,7 @@ tests += {
't/011_lock_stats.pl',
't/012_ddlutils.pl',
't/013_temp_obj_multisession.pl',
+ 't/014_pg_log_query_plan.pl',
],
# The injection points are cluster-wide, so disable installcheck
'runningcheck': false,
diff --git a/src/test/modules/test_misc/t/014_pg_log_query_plan.pl b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
new file mode 100644
index 00000000000..2055641283e
--- /dev/null
+++ b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
@@ -0,0 +1,101 @@
+
+# Copyright (c) 2024-2026, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+use locale;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test that pg_log_query_plan() actually logs the query plan of
+# another backend executing a query.
+
+# This test requires timing coordinations:
+# 1) The target backend must be executing a query when
+# pg_log_query_plan() sends the signal.
+# 2) We must confirm that the target backend actually received the
+# signal that requests logging of the plan.
+#
+# We use an advisory lock and an injection point to control them
+# respectively.
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+my $psql_session1 = $node->background_psql('postgres');
+my $psql_session2 = $node->background_psql('postgres');
+
+my $session1_pid = $psql_session1->query_safe("select pg_backend_pid()");
+
+# Set injection point in the logging plan request handler to ensure
+# that session1 received the signal of pg_log_query_plan().
+$psql_session1->query_safe(
+ qq[
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('log-query-interrupt', 'wait');
+]);
+
+# Use an advisory lock to make session1 blocked during query execution.
+$psql_session2->query_safe(
+ qq[
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+]);
+
+$psql_session1->query_until(
+ qr/wait_on_advisory_lock/, q(
+ \echo wait_on_advisory_lock
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+));
+
+# Confirm that session1 is actually waiting on the advisory lock.
+$node->wait_for_event('client backend', 'advisory');
+
+# Run pg_log_query_plan().
+$psql_session2->query_safe("SELECT pg_log_query_plan($session1_pid);");
+
+# Ensure that the signal of pg_log_query_plan() is actually
+# received by confirming session1 is waiting on the injection point.
+$node->wait_for_event('client backend', 'log-query-interrupt');
+
+# Commit the session 2 to release the advisory lock.
+$psql_session2->query_safe("COMMIT;");
+
+my $log_offset = -s $node->logfile;
+
+# Detach the injection point to start logging the plan.
+$psql_session2->query_safe(
+ qq[
+ SELECT injection_points_wakeup('log-query-interrupt');
+ SELECT injection_points_detach('log-query-interrupt');
+]);
+
+$node->wait_for_log('query and its plan running on backend with PID',
+ $log_offset);
+
+$psql_session1->query_safe("COMMIT;");
+
+ok($psql_session1->quit);
+ok($psql_session2->quit);
+
+done_testing();
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3261bff209..5d310939a75 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -357,6 +357,44 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 946ee5726cd..e7ab81cca60 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -113,6 +113,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/009_pg_log_query_plan.pl.
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2026-06-22 13:50 ` Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2026-06-22 13:50 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Mon, Jun 15, 2026 at 9:26 AM torikoshia <[email protected]> wrote:
> I also fixed some typos in the attached patch.
In my earlier reviews, I have been focusing on core correctness
issues. This time, I decided to leave that aside to focus on a few
more cosmetic and user-interface points:
+#ifdef USE_INJECTION_POINTS
+ INJECTION_POINT("log-query-interrupt", NULL);
+#endif
I think the #ifdef is not required here. Look at how INJECTION_POINT()
is defined.
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
So, we're establishing a non-default set of settings here which the
user has no way to configure. I think that's hard to justify. I think
we either need to accept the defaults that NewExplainState()
configures, or we need to provide a GUC for the user to configure
things as they wish.
+++ b/src/backend/commands/dynamic_explain.c
We have four explain-related source files currently, all of them have
"explain" at the start of the name rather than at the end. We should
probably do the same here. I would suggest explain_running.c rather
than explain_dynamic.c. Or some other word that conveys "the query is
in progress" -- dynamic seems vague.
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
It's worth thinking about whether restricting this to the superuser by
default is the right decision. I don't think I believe this rationale:
a user can do lots of things that generate lots of log volume, and we
don't restrict any of the other ones. For example, any user can do
this:
do $$ begin for i in 1..1000000 loop raise log '% is an extremely long
string', repeat('hoge',100000); end loop; end; $$;
I thought about whether we should actually relax this and let people
log their own queries, but then I realized what I believe the real
issue to be: the superuser is presumed to have access to the log files
-- after all, they can configure them, and escape to the shell -- but
other users very likely don't. If some of them do, the superuser knows
which ones.
+extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
There's no such function (any more, anyway).
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2026-06-24 12:56 ` torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2026-06-24 12:56 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Mon, Jun 22, 2026 at 10:51 PM Robert Haas <[email protected]>
wrote:
>
> On Mon, Jun 15, 2026 at 9:26 AM torikoshia <[email protected]>
> wrote:
> > I also fixed some typos in the attached patch.
>
> In my earlier reviews, I have been focusing on core correctness
> issues. This time, I decided to leave that aside to focus on a few
> more cosmetic and user-interface points:
Thank you for the review!
> +#ifdef USE_INJECTION_POINTS
> + INJECTION_POINT("log-query-interrupt", NULL);
> +#endif
>
> I think the #ifdef is not required here. Look at how INJECTION_POINT()
> is defined.
Agreed.
> + es = NewExplainState();
> +
> + es->format = EXPLAIN_FORMAT_TEXT;
> + es->settings = true;
> + es->verbose = true;
> + es->signaled = true;
>
> So, we're establishing a non-default set of settings here which the
> user has no way to configure. I think that's hard to justify. I think
> we either need to accept the defaults that NewExplainState()
> configures, or we need to provide a GUC for the user to configure
> things as they wish.
At this stage, what I would like to focus on is developing the mechanism
for obtaining the plan of a currently running query. So I think it is
better to start with the default EXPLAIN options.
Updated the patch that way.
> +++ b/src/backend/commands/dynamic_explain.c
>
> We have four explain-related source files currently, all of them have
> "explain" at the start of the name rather than at the end. We should
> probably do the same here. I would suggest explain_running.c rather
> than explain_dynamic.c. Or some other word that conveys "the query is
> in progress" -- dynamic seems vague.
Thanks for the suggestion. Renamed it to explain_running.c.
> + * By default, only superusers are allowed to signal to log the plan
> because
> + * allowing any users to issue this request at an unbounded rate would
> + * cause lots of log messages and which can lead to denial of service.
>
> It's worth thinking about whether restricting this to the superuser by
> default is the right decision. I don't think I believe this rationale:
> a user can do lots of things that generate lots of log volume, and we
> don't restrict any of the other ones. For example, any user can do
> this:
>
> do $$ begin for i in 1..1000000 loop raise log '% is an extremely long
> string', repeat('hoge',100000); end loop; end; $$;
>
> I thought about whether we should actually relax this and let people
> log their own queries, but then I realized what I believe the real
> issue to be: the superuser is presumed to have access to the log files
> -- after all, they can configure them, and escape to the shell -- but
> other users very likely don't. If some of them do, the superuser knows
> which ones.
That makes sense.
Even if a non-superuser has EXECUTE permission on this function, it
would
probably not be very useful, because the user would not be able to read
the server log by themselves as you pointed out.
Would it make sense to restrict EXECUTE to superusers by default for
that
reason?
In the attached patch, I have rewritten the comment as follows:
+ * By default, only superusers are allowed to signal a backend to log
its
+ * plan because the output is written to the server log, which in
many cases
+ * can only be read by superusers.
The current restriction was modeled after
pg_log_backend_memory_contexts().
So I think the same discussion would also apply to that function. If
needed,
we could also revisit the comment there from the same perspective:
* pg_log_backend_memory_contexts
* Signal a backend or an auxiliary process to log its memory
contexts.
*
* By default, only superusers are allowed to signal to log the memory
* contexts because allowing any users to issue this request at an
unbounded
* rate would cause lots of log messages and which can lead to denial
of
* service. Additional roles can be permitted with GRANT.
BTW I'm interested in the proposal of
pg_get_process_memory_contexts()[1],
which would return memory context information directly from the function
rather than writing it to the log. If a similar approach were adopted
for running query plans in the future, then it would make sense to relax
the default privilege for that.
> +extern TupleTableSlot *ExecProcNodeWithExplain(PlanState *ps);
>
> There's no such function (any more, anyway).
Removed.
[1]
https://www.postgresql.org/message-id/495F9D76-6B8B-45C3-95DA-EF5A8762DA19%40yesql.se
Thanks,
--
Atsushi Torikoshi
Seconded from NTT DATA CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v56-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.1K, ../../[email protected]/2-v56-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From 34d678ca3c424d1e8f3b33938547b4b77e1cadda Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 24 Jun 2026 21:30:01 +0900
Subject: [PATCH v56] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implelements logging query plan.
When executor executes the one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 29 +--
doc/src/sgml/func/func-admin.sgml | 24 +++
src/backend/access/transam/xact.c | 34 ++++
src/backend/commands/Makefile | 1 +
src/backend/commands/explain.c | 44 ++++-
src/backend/commands/explain_running.c | 182 ++++++++++++++++++
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 121 +++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 7 +
src/include/commands/explain.h | 5 +
src/include/commands/explain_running.h | 24 +++
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/modules/test_misc/meson.build | 1 +
.../test_misc/t/014_pg_log_query_plan.pl | 101 ++++++++++
src/test/regress/expected/misc_functions.out | 38 ++++
src/test/regress/sql/misc_functions.sql | 31 +++
24 files changed, 641 insertions(+), 37 deletions(-)
create mode 100644 src/backend/commands/explain_running.c
create mode 100644 src/include/commands/explain_running.h
create mode 100644 src/test/modules/test_misc/t/014_pg_log_query_plan.pl
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 97ce498d0c1..07014cab366 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -451,32 +451,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
apply_extension_options(es, extension_options);
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- if (explain_per_plan_hook)
- (*explain_per_plan_hook) (queryDesc->plannedstmt,
- NULL, es,
- queryDesc->sourceText,
- queryDesc->params,
- queryDesc->estate->es_queryEnv);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 0eae1c1f616..ba319c94160 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,30 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index de4cf96eaa2..cc37ca565f1 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -37,6 +37,7 @@
#include "catalog/pg_enum.h"
#include "catalog/storage.h"
#include "commands/async.h"
+#include "commands/explain_running.h"
#include "commands/tablecmds.h"
#include "commands/trigger.h"
#include "common/pg_prng.h"
@@ -217,6 +218,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -250,6 +252,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -935,6 +938,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2953,6 +2977,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current query plan state. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5352,6 +5379,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current query plan state. Note that even after this reset, it's
+ * still possible to obtain the parent transaction's query plans, since
+ * they are preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 5b9d084977e..671f036fa76 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -35,6 +35,7 @@ OBJS = \
explain.o \
explain_dr.o \
explain_format.o \
+ explain_running.o \
explain_state.o \
extension.o \
foreigncmds.o \
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index a40d03d35f3..2221b8e9569 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1092,6 +1092,43 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ if (explain_per_plan_hook)
+ (*explain_per_plan_hook) (queryDesc->plannedstmt,
+ NULL, es,
+ queryDesc->sourceText,
+ queryDesc->params,
+ queryDesc->estate->es_queryEnv);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1827,7 +1864,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1875,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/explain_running.c b/src/backend/commands/explain_running.c
new file mode 100644
index 00000000000..0fe2ae3f191
--- /dev/null
+++ b/src/backend/commands/explain_running.c
@@ -0,0 +1,182 @@
+/*-------------------------------------------------------------------------
+ *
+ * explain_running.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/explain_running.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_running.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "storage/procsignal.h"
+#include "utils/backend_status.h"
+#include "utils/injection_point.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ INJECTION_POINT("log-query-interrupt", NULL);
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward(i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ if (queryDesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ ExplainStringAssemble(es, queryDesc, es->format, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ LogQueryPlanPending = false;
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly
+ * and rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal a backend to log its
+ * plan because the output is written to the server log, which in many cases
+ * can only be read by superusers.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index 9f258d566eb..d630dedd5d7 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -23,6 +23,7 @@ backend_sources += files(
'explain.c',
'explain_dr.c',
'explain_format.c',
+ 'explain_running.c',
'explain_state.c',
'extension.c',
'foreigncmds.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4b30f768680..cca01d3c4fc 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -44,6 +44,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/explain_running.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -323,6 +324,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -335,6 +337,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -397,6 +406,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStop(queryDesc->query_instr);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Otherwise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 7c4c66e323f..109016d0602 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/explain_running.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeAgg.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,21 +443,37 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
@@ -527,6 +545,91 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 1397f65f67b..4b442d4cdad 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/explain_running.h"
#include "commands/repack.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -713,6 +714,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index dbef734a93f..7f8790d340a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/explain_running.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
#include "commands/repack.h"
@@ -3609,6 +3610,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index bbd28d14d99..de84fd9cfa3 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index a8cbdf247c8..f2881d956c0 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+QueryDesc *GetCurrentQueryDesc(void);
+void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern int GetTopReadOnlyTransactionNestLevel(void);
extern void CommandCounterIncrement(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa76c7923f0..dba77450b22 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8721,6 +8721,13 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts', proacl => '{POSTGRES=X}' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan',
+ proacl => '{POSTGRES=X}' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 472e141bba3..42f1f7d61ff 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -71,6 +71,8 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const BufferUsage *bufusage,
const MemoryContextCounters *mem_counters);
+extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
+extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es,
QueryDesc *queryDesc);
@@ -81,5 +83,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_running.h b/src/include/commands/explain_running.h
new file mode 100644
index 00000000000..316ef25be60
--- /dev/null
+++ b/src/include/commands/explain_running.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * explain_running.h
+ * prototypes for explain_running.c
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/explain_running.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef EXPLAIN_RUNNING_H
+#define EXPLAIN_RUNNING_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* EXPLAIN_RUNNING_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 97bc7ed49f6..b9a98157815 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -73,6 +73,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 650baab3efc..12a1673d00c 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -299,6 +299,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 7170a4bff98..0ab775e964f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -98,6 +98,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index aaa158bfd66..c2142043e2d 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
PROCSIG_SLOTSYNC_MESSAGE, /* ask slot synchronization to stop */
PROCSIG_REPACK_MESSAGE, /* Message from repack worker */
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index 969e90b396d..4c23bb794c7 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -22,6 +22,7 @@ tests += {
't/011_lock_stats.pl',
't/012_ddlutils.pl',
't/013_temp_obj_multisession.pl',
+ 't/014_pg_log_query_plan.pl',
],
# The injection points are cluster-wide, so disable installcheck
'runningcheck': false,
diff --git a/src/test/modules/test_misc/t/014_pg_log_query_plan.pl b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
new file mode 100644
index 00000000000..2055641283e
--- /dev/null
+++ b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
@@ -0,0 +1,101 @@
+
+# Copyright (c) 2024-2026, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+use locale;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test that pg_log_query_plan() actually logs the query plan of
+# another backend executing a query.
+
+# This test requires timing coordinations:
+# 1) The target backend must be executing a query when
+# pg_log_query_plan() sends the signal.
+# 2) We must confirm that the target backend actually received the
+# signal that requests logging of the plan.
+#
+# We use an advisory lock and an injection point to control them
+# respectively.
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+my $psql_session1 = $node->background_psql('postgres');
+my $psql_session2 = $node->background_psql('postgres');
+
+my $session1_pid = $psql_session1->query_safe("select pg_backend_pid()");
+
+# Set injection point in the logging plan request handler to ensure
+# that session1 received the signal of pg_log_query_plan().
+$psql_session1->query_safe(
+ qq[
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('log-query-interrupt', 'wait');
+]);
+
+# Use an advisory lock to make session1 blocked during query execution.
+$psql_session2->query_safe(
+ qq[
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+]);
+
+$psql_session1->query_until(
+ qr/wait_on_advisory_lock/, q(
+ \echo wait_on_advisory_lock
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+));
+
+# Confirm that session1 is actually waiting on the advisory lock.
+$node->wait_for_event('client backend', 'advisory');
+
+# Run pg_log_query_plan().
+$psql_session2->query_safe("SELECT pg_log_query_plan($session1_pid);");
+
+# Ensure that the signal of pg_log_query_plan() is actually
+# received by confirming session1 is waiting on the injection point.
+$node->wait_for_event('client backend', 'log-query-interrupt');
+
+# Commit the session 2 to release the advisory lock.
+$psql_session2->query_safe("COMMIT;");
+
+my $log_offset = -s $node->logfile;
+
+# Detach the injection point to start logging the plan.
+$psql_session2->query_safe(
+ qq[
+ SELECT injection_points_wakeup('log-query-interrupt');
+ SELECT injection_points_detach('log-query-interrupt');
+]);
+
+$node->wait_for_log('query and its plan running on backend with PID',
+ $log_offset);
+
+$psql_session1->query_safe("COMMIT;");
+
+ok($psql_session1->quit);
+ok($psql_session2->quit);
+
+done_testing();
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3261bff209..30297d7bd4e 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -357,6 +357,44 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/014_pg_log_query_plan.pl.
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 946ee5726cd..d3cf5b6f852 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -113,6 +113,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/014_pg_log_query_plan.pl.
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2026-06-24 14:35 ` Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Andrei Lepikhov @ 2026-06-24 14:35 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; Robert Haas <[email protected]>; [email protected]; [email protected]
Hi,
I began discovering how it works.
Some minor issues after skimming through the code:
1. pg_log_query_plan - This function just sends a signal. There are no
guarantees that actual action will be performed. So, the 'request' word makes
more sense for me.
2. GetCurrentQueryDesc, SetCurrentQueryDesc - need 'extern' in the .h file
3. ExplainPrintPlan, ExplainPrintTriggers - have exact duplicates in the explain.h
4. ExplainStringAssemble() has '0' input for a boolean parameter.
5. ExecSetExecProcNodeRecurse: I don't see PlanState::initPlan. Is this intentional?
6. INJECTION_POINT("log-query-interrupt", ...) - it is called inside a
signal-safe zone. In tests, the caller might do a lot of things in the attached
routine, causing hidden problems later.
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
@ 2026-06-25 09:43 ` Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2026-06-25 09:43 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; +Cc: torikoshia <[email protected]>; Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Wed, Jun 24, 2026 at 10:35 AM Andrei Lepikhov <[email protected]> wrote:
> 1. pg_log_query_plan - This function just sends a signal. There are no
> guarantees that actual action will be performed. So, the 'request' word makes
> more sense for me.
I don't agree with this particular comment. Of course nothing is
absolutely guaranteed because the database system could crash or the
world could end, but if the target backend is running a query, it
should log the query plan. Therefore, I don't see the need for a word
like "request".
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2026-06-25 10:33 ` Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Andrei Lepikhov @ 2026-06-25 10:33 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: torikoshia <[email protected]>; Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On 25/06/2026 11:43, Robert Haas wrote:
> On Wed, Jun 24, 2026 at 10:35 AM Andrei Lepikhov <[email protected]> wrote:
>> 1. pg_log_query_plan - This function just sends a signal. There are no
>> guarantees that actual action will be performed. So, the 'request' word makes
>> more sense for me.
>
> I don't agree with this particular comment. Of course nothing is
> absolutely guaranteed because the database system could crash or the
> world could end, but if the target backend is running a query, it
> should log the query plan. Therefore, I don't see the need for a word
> like "request".
>
Ok, so just mention this behaviour in the documentation - let people know that
they should potentially wait for a minute or two more to see the EXPLAIN.
Real-life with huge queries and big machines provide us with examples where a
backend might delay a response to a signal for quite a substantial time.
Sometimes nothing happens after such an async operation, and we need to identify
the problem: has the backend stalled, is the ‘logging plan’ algorithm
ineffective, or is something else happening?
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
@ 2026-06-26 12:50 ` Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Robert Haas @ 2026-06-26 12:50 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; +Cc: torikoshia <[email protected]>; Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Thu, Jun 25, 2026 at 6:33 AM Andrei Lepikhov <[email protected]> wrote:
> Ok, so just mention this behaviour in the documentation - let people know that
> they should potentially wait for a minute or two more to see the EXPLAIN.
> Real-life with huge queries and big machines provide us with examples where a
> backend might delay a response to a signal for quite a substantial time.
>
> Sometimes nothing happens after such an async operation, and we need to identify
> the problem: has the backend stalled, is the ‘logging plan’ algorithm
> ineffective, or is something else happening?
I don't think we document this in other, similar cases. Many things
can be delayed if the machine is overloaded, but unless I am missing
something, having to wait over a minute for this to work would be
*extremely* unusual and only happen in case of a machine that is
absolutely being crushed by the load. And in that case everything else
will be slow, too.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
@ 2026-06-29 04:23 ` torikoshia <[email protected]>
2026-06-29 16:30 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-07-01 10:42 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: torikoshia @ 2026-06-29 04:23 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; Robert Haas <[email protected]>; +Cc: Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Wed, Jun 24, 2026 at 11:35 PM Andrei Lepikhov <[email protected]>
wrote:
Thanks for your review!
> 2. GetCurrentQueryDesc, SetCurrentQueryDesc - need 'extern' in the .h
> file
> 3. ExplainPrintPlan, ExplainPrintTriggers - have exact duplicates in
> the explain.h
> 4. ExplainStringAssemble() has '0' input for a boolean parameter.
> 5. ExecSetExecProcNodeRecurse: I don't see PlanState::initPlan. Is this
> intentional?
> 6. INJECTION_POINT("log-query-interrupt", ...) - it is called inside a
> signal-safe zone. In tests, the caller might do a lot of things in the
> attached
> routine, causing hidden problems later.
I agree that these points should be addressed.
I'll post an updated patch.
I've also received some off-list feedback, and I'm planning to
incorporate
those comments into the updated patch as well.
On 2026-06-26 21:50, Robert Haas wrote:
> On Thu, Jun 25, 2026 at 6:33 AM Andrei Lepikhov <[email protected]>
> wrote:
>> Ok, so just mention this behaviour in the documentation - let people
>> know that
>> they should potentially wait for a minute or two more to see the
>> EXPLAIN.
>> Real-life with huge queries and big machines provide us with examples
>> where a
>> backend might delay a response to a signal for quite a substantial
>> time.
>>
>> Sometimes nothing happens after such an async operation, and we need
>> to identify
>> the problem: has the backend stalled, is the ‘logging plan’ algorithm
>> ineffective, or is something else happening?
>
> I don't think we document this in other, similar cases. Many things
> can be delayed if the machine is overloaded, but unless I am missing
> something, having to wait over a minute for this to work would be
> *extremely* unusual and only happen in case of a machine that is
> absolutely being crushed by the load. And in that case everything else
> will be slow, too.
I think a delay caused by signal handling can also occur with the
existing
functions, such as pg_cancel_backend() and
pg_log_backend_memory_contexts().
Since the documentation does not specifically mention such delays for
those
functions, I thought it may not be necessary to document this point for
pg_log_query_plan() either.
On the other hand, since pg_log_query_plan() emits the log output at the
next time ExecProcNodeFirst() is executed, if there is a delay before
ExecProcNodeFirst() is called, there would also be a delay before the
plan is logged.
For example, this can happen with a query such as SELECT pg_sleep(100),
or when executing a time-consuming sort operation like the following:
BEGIN;
DECLARE sort_probe NO SCROLL CURSOR FOR
SELECT g, md5(g::text) AS k
FROM generate_series(1, 100000000) AS g
ORDER BY k;
FETCH 1 FROM sort_probe;
I think this behavior is specific to pg_log_query_plan(), so I am
starting to wonder whether something should be mentioned in the
documentation.
Thanks,
--
Atsushi Torikoshi
Seconded from NTT DATA CORPORATION to SRA OSS K.K.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2026-06-29 16:30 ` Robert Haas <[email protected]>
1 sibling, 0 replies; 124+ messages in thread
From: Robert Haas @ 2026-06-29 16:30 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Andrei Lepikhov <[email protected]>; Lukas Fittl <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Mon, Jun 29, 2026 at 12:23 AM torikoshia <[email protected]> wrote:
> I think this behavior is specific to pg_log_query_plan(), so I am
> starting to wonder whether something should be mentioned in the
> documentation.
Yeah, that's fair. In that case, the delay could be longer than what
people might expect, so it could be worth calling out for that reason.
It still feels optional to me, but I'm fine if you want to mention it.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2026-07-01 10:42 ` Andrei Lepikhov <[email protected]>
2026-07-01 13:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Andrei Lepikhov @ 2026-07-01 10:42 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Robert Haas <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On 29/06/2026 06:23, torikoshia wrote:
> I think this behavior is specific to pg_log_query_plan(), so I am
> starting to wonder whether something should be mentioned in the
> documentation.
I took a closer look at the solution, and it looks quite good.
1. EXPLAIN needs a more or less consistent execution plan state. And this patch
decides to cut in the executor right before a node should start producing the
next tuple.
I’ve never seen any races or other issues with this approach in Postgres forks,
except that users sometimes felt irritated because a HashJoin gets stuck
rebalancing hash batches and does not respond for a long time, or when an OLAP
query causes a situation close to OOM, and the user can’t get an explain because
of a massive memory operation.
The bad example of such a feature can be found in the AQO project [1]. In this
first attempt, we used the built-in RegisterTimeout() to stop execution and take
an EXPLAIN. And from time to time, it fails due to the executor's inconsistent
state. So, the current way looks right.
2. Adding another event processing type to the ProcessInterrupts() routine is a
brilliant idea. This way, overhead is only added when someone requests an
injection into the execution process.
3. But the idea of the ExecSetExecProcNodeRecurse() seems fragile to me. It
depends on the PlanState tree structure and a choice on the current QueryDesc.
When adding a new node, we will need to remember to update this code, too. I’d
propose introducing a tiny hook (register callback?) into ExecProcNode() and
just setting it in ProcessLogQueryPlanInterrupt. Next call of the ExecProcNode()
will cause the EXPLAIN machinery whenever it happens. It might also simplify
this function's code, since races won't be a danger anymore. Just don't forget
to restore the hook state after the EXPLAIN preparation or top-level query end.
In addition, such ExecProcNode_hook may serve the longstanding desire of
extension developers to look inside the execution of a query: large queries run
for seconds, minutes, or even longer. And it is quite a frequent dilemma: wait a
little more or interrupt and replan the query? Even with this feature being
committed, we still might want to get an EXPLAIN of the top-level query, not a
nested one, that the GetCurrentQueryDesc provides us with. Exactly this way was
implemented in the replan feature [2] - unfortunately, it is still closed source
code.
4. Also, I’m not sure it is safe to store the QueryDesc pointer at all at the
ProcessLogQueryPlanInterrupt. Maybe the next call to ExecProcNode() will be made
from an external query (or from a deeper one). The current implementation will
not produce any EXPLAIN in this case, but query still executes.
I think, combination of a global hook and queryDesc pointer, saved somewhere in
the EState may solve the problem. It is available at each executor's node, and
the EXPLAIN machinery might use an exact descriptor related to the currently
executing query.
5. The phrase ‘only the plan of the most deeply nested query is logged’ is
missed in the docs. So the current decision has not yet been documented.
[1]
https://github.com/postgrespro/aqo/blob/4a7fcd7291a8c4209c9102d1736f9f754d311cf2/postprocessing.c#L6...
[2] https://postgrespro.com/docs/enterprise/16/realtime-query-replanning
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-01 10:42 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
@ 2026-07-01 13:09 ` torikoshia <[email protected]>
2026-07-02 11:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2026-07-01 13:09 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Robert Haas <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Mon, Jun 29, 2026 at 1:23 PM torikoshia <[email protected]>
wrote:
>
> On Wed, Jun 24, 2026 at 11:35 PM Andrei Lepikhov <[email protected]>
> wrote:
> Thanks for your review!
>
> > 2. GetCurrentQueryDesc, SetCurrentQueryDesc - need 'extern' in the .h
> > file
> > 3. ExplainPrintPlan, ExplainPrintTriggers - have exact duplicates in
> > the explain.h
> > 4. ExplainStringAssemble() has '0' input for a boolean parameter.
> > 5. ExecSetExecProcNodeRecurse: I don't see PlanState::initPlan. Is this
> > intentional?
> > 6. INJECTION_POINT("log-query-interrupt", ...) - it is called inside a
> > signal-safe zone. In tests, the caller might do a lot of things in the
> > attached
> > routine, causing hidden problems later.
>
> I agree that these points should be addressed.
> I'll post an updated patch.
Fixed these points.
> I've also received some off-list feedback, and I'm planning to
> incorporate those comments into the updated patch as well.
I also made the following changes:
- Updated the documentation to mention that pg_log_query_plan() is
asynchronous and that plan logging can be delayed if the target backend
does not reach an executor point where the plan can be inspected.
- Fixed LogQueryPlan() so that its temporary memory context is always
restored and deleted, even when there is no current QueryDesc.
- Added defensive checks for pgstat_get_beentry_by_proc_number()
returning NULL, and verified that the backend status entry still matches
the requested PID.
- Wrapped the main LogQueryPlan() body in PG_TRY/PG_FINALLY so
LogQueryPlanPending and memory context cleanup are handled even if
EXPLAIN generation throws an error.
On Wed, Jul 1, 2026 at 7:42 PM Andrei Lepikhov <[email protected]>
wrote:
> I took a closer look at the solution
Thanks for the comments!
I will mainly consider the 3rd and 4th points.
Regards,
--
Atsushi Torikoshi
Seconded from NTT DATA CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v57-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.1K, ../../[email protected]/2-v57-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From d518511ec057aae29ac4edd38dfda28b373d3dc8 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 1 Jul 2026 21:37:47 +0900
Subject: [PATCH v57] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implements logging query plan.
When the executor next invokes one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 29 +--
doc/src/sgml/func/func-admin.sgml | 32 +++
src/backend/access/transam/xact.c | 33 ++++
src/backend/commands/Makefile | 1 +
src/backend/commands/explain.c | 44 ++++-
src/backend/commands/explain_running.c | 187 ++++++++++++++++++
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 132 ++++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 7 +
src/include/commands/explain.h | 3 +
src/include/commands/explain_running.h | 24 +++
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/modules/test_misc/meson.build | 1 +
.../test_misc/t/014_pg_log_query_plan.pl | 101 ++++++++++
src/test/regress/expected/misc_functions.out | 38 ++++
src/test/regress/sql/misc_functions.sql | 31 +++
24 files changed, 662 insertions(+), 37 deletions(-)
create mode 100644 src/backend/commands/explain_running.c
create mode 100644 src/include/commands/explain_running.h
create mode 100644 src/test/modules/test_misc/t/014_pg_log_query_plan.pl
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 97ce498d0c1..07014cab366 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -451,32 +451,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
apply_extension_options(es, extension_options);
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- if (explain_per_plan_hook)
- (*explain_per_plan_hook) (queryDesc->plannedstmt,
- NULL, es,
- queryDesc->sourceText,
- queryDesc->params,
- queryDesc->estate->es_queryEnv);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 0eae1c1f616..877d75d9c5e 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,38 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ The plan is logged when the target backend next reaches a
+ point where the running plan can be inspected. If the backend
+ is spending a long time in work that does not reach such a
+ point, the log output can be delayed until that work
+ completes. If the query finishes before such a point is
+ reached, no plan may be logged.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index de4cf96eaa2..e130afe2c11 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -250,6 +251,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -935,6 +937,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2953,6 +2976,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current QueryDesc. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5352,6 +5378,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current QueryDesc. Note that even after this reset, it's still
+ * possible to obtain the parent transaction's query plans, since they are
+ * preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 5b9d084977e..671f036fa76 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -35,6 +35,7 @@ OBJS = \
explain.o \
explain_dr.o \
explain_format.o \
+ explain_running.o \
explain_state.o \
extension.o \
foreigncmds.o \
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index a40d03d35f3..2221b8e9569 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1092,6 +1092,43 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ if (explain_per_plan_hook)
+ (*explain_per_plan_hook) (queryDesc->plannedstmt,
+ NULL, es,
+ queryDesc->sourceText,
+ queryDesc->params,
+ queryDesc->estate->es_queryEnv);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1827,7 +1864,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when the query has already been executed and explain
+ * has been requested by signal, as the target query may use
+ * instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1875,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/explain_running.c b/src/backend/commands/explain_running.c
new file mode 100644
index 00000000000..ecc9f28e60f
--- /dev/null
+++ b/src/backend/commands/explain_running.c
@@ -0,0 +1,187 @@
+/*-------------------------------------------------------------------------
+ *
+ * explain_running.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/explain_running.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_running.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "storage/procsignal.h"
+#include "utils/backend_status.h"
+#include "utils/injection_point.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+ es->signaled = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward (i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ PG_TRY();
+ {
+ if (queryDesc != NULL)
+ {
+ ExplainStringAssemble(es, queryDesc, es->format, false, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+ }
+ }
+ PG_FINALLY();
+ {
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+ LogQueryPlanPending = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly
+ * and rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ INJECTION_POINT("log-query-interrupt", NULL);
+
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal a backend to log its
+ * plan because the output is written to the server log, which in many cases
+ * can only be read by superusers.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status = NULL;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc != NULL)
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+
+ if (proc == NULL || be_status == NULL || be_status->st_procpid != pid)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index 9f258d566eb..d630dedd5d7 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -23,6 +23,7 @@ backend_sources += files(
'explain.c',
'explain_dr.c',
'explain_format.c',
+ 'explain_running.c',
'explain_state.c',
'extension.c',
'foreigncmds.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4b30f768680..cca01d3c4fc 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -44,6 +44,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/explain_running.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -323,6 +324,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -335,6 +337,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -397,6 +406,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStop(queryDesc->query_instr);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Otherwise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 7c4c66e323f..b8c118dea4f 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/explain_running.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeAgg.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,21 +443,37 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
@@ -527,6 +545,102 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * initPlan, subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->initPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->initPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 1397f65f67b..4b442d4cdad 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/explain_running.h"
#include "commands/repack.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -713,6 +714,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index dbef734a93f..7f8790d340a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/explain_running.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
#include "commands/repack.h"
@@ -3609,6 +3610,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index bbd28d14d99..de84fd9cfa3 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index a8cbdf247c8..0570fdd3277 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+extern QueryDesc *GetCurrentQueryDesc(void);
+extern void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern int GetTopReadOnlyTransactionNestLevel(void);
extern void CommandCounterIncrement(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa76c7923f0..dba77450b22 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8721,6 +8721,13 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts', proacl => '{POSTGRES=X}' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan',
+ proacl => '{POSTGRES=X}' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 472e141bba3..633ac94ea58 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -81,5 +81,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_running.h b/src/include/commands/explain_running.h
new file mode 100644
index 00000000000..316ef25be60
--- /dev/null
+++ b/src/include/commands/explain_running.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * explain_running.h
+ * prototypes for explain_running.c
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/explain_running.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef EXPLAIN_RUNNING_H
+#define EXPLAIN_RUNNING_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* EXPLAIN_RUNNING_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 97bc7ed49f6..b9a98157815 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -73,6 +73,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 650baab3efc..12a1673d00c 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -299,6 +299,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 7170a4bff98..0ab775e964f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -98,6 +98,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index aaa158bfd66..c2142043e2d 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
PROCSIG_SLOTSYNC_MESSAGE, /* ask slot synchronization to stop */
PROCSIG_REPACK_MESSAGE, /* Message from repack worker */
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index 969e90b396d..4c23bb794c7 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -22,6 +22,7 @@ tests += {
't/011_lock_stats.pl',
't/012_ddlutils.pl',
't/013_temp_obj_multisession.pl',
+ 't/014_pg_log_query_plan.pl',
],
# The injection points are cluster-wide, so disable installcheck
'runningcheck': false,
diff --git a/src/test/modules/test_misc/t/014_pg_log_query_plan.pl b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
new file mode 100644
index 00000000000..2055641283e
--- /dev/null
+++ b/src/test/modules/test_misc/t/014_pg_log_query_plan.pl
@@ -0,0 +1,101 @@
+
+# Copyright (c) 2024-2026, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+use locale;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test that pg_log_query_plan() actually logs the query plan of
+# another backend executing a query.
+
+# This test requires timing coordinations:
+# 1) The target backend must be executing a query when
+# pg_log_query_plan() sends the signal.
+# 2) We must confirm that the target backend actually received the
+# signal that requests logging of the plan.
+#
+# We use an advisory lock and an injection point to control them
+# respectively.
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+my $psql_session1 = $node->background_psql('postgres');
+my $psql_session2 = $node->background_psql('postgres');
+
+my $session1_pid = $psql_session1->query_safe("select pg_backend_pid()");
+
+# Set injection point in the logging plan request handler to ensure
+# that session1 received the signal of pg_log_query_plan().
+$psql_session1->query_safe(
+ qq[
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('log-query-interrupt', 'wait');
+]);
+
+# Use an advisory lock to make session1 blocked during query execution.
+$psql_session2->query_safe(
+ qq[
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+]);
+
+$psql_session1->query_until(
+ qr/wait_on_advisory_lock/, q(
+ \echo wait_on_advisory_lock
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+));
+
+# Confirm that session1 is actually waiting on the advisory lock.
+$node->wait_for_event('client backend', 'advisory');
+
+# Run pg_log_query_plan().
+$psql_session2->query_safe("SELECT pg_log_query_plan($session1_pid);");
+
+# Ensure that the signal of pg_log_query_plan() is actually
+# received by confirming session1 is waiting on the injection point.
+$node->wait_for_event('client backend', 'log-query-interrupt');
+
+# Commit the session 2 to release the advisory lock.
+$psql_session2->query_safe("COMMIT;");
+
+my $log_offset = -s $node->logfile;
+
+# Detach the injection point to start logging the plan.
+$psql_session2->query_safe(
+ qq[
+ SELECT injection_points_wakeup('log-query-interrupt');
+ SELECT injection_points_detach('log-query-interrupt');
+]);
+
+$node->wait_for_log('query and its plan running on backend with PID',
+ $log_offset);
+
+$psql_session1->query_safe("COMMIT;");
+
+ok($psql_session1->quit);
+ok($psql_session2->quit);
+
+done_testing();
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3261bff209..30297d7bd4e 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -357,6 +357,44 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/014_pg_log_query_plan.pl.
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 946ee5726cd..d3cf5b6f852 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -113,6 +113,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/014_pg_log_query_plan.pl.
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-01 10:42 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-01 13:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2026-07-02 11:05 ` Andrei Lepikhov <[email protected]>
2026-07-06 06:21 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Andrei Lepikhov @ 2026-07-02 11:05 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Robert Haas <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On 01/07/2026 15:09, torikoshia wrote:
> Thanks for the comments!
> I will mainly consider the 3rd and 4th points.
Thanks,
I read the EXPLAIN generation part of the code. It looks good.
It prepares a clean EXPLAIN that’s good for stability and safety, because the
hardest part of the task has been correctly gathering the instrumentation for
the partially executed query: interrupting parallel workers and merging their
instrumentation is always the pain.
Also, it leaves room for extensions or future development if the user wants to
make a hard stop and take a full picture of the current state, including rows,
buffers, WAL, and whatever is possible with current instrumentation.
The only 'es→signaled' flag is an eyesore. It would be better to base the
decision on the es→analyze flag, which would break the long-standing
auto_explain rule, if I understand correctly. So, at least this flag should be
renamed to something like es→running, or es→query_in_progress.
Also, not sure I understand how this code regulates the explain format,
verbosity and other EXPLAIN settings.
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-01 10:42 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-01 13:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-02 11:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
@ 2026-07-06 06:21 ` torikoshia <[email protected]>
2026-07-06 13:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2026-07-06 06:21 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Robert Haas <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On Wed, Jul 1, 2026 at 7:42 PM Andrei Lepikhov <[email protected]>
wrote:
> I’d
> propose introducing a tiny hook (register callback?) into
> ExecProcNode() and
> just setting it in ProcessLogQueryPlanInterrupt.
This approach would also be possible, but we are concerned that
checking the hook on every ExecProcNode() call would add too much
overhead and could have a significant performance impact.
This point was discussed previously:
https://www.postgresql.org/message-id/20240215185911.v4o6fo444md6a3w7%40awork3.anarazel.de
> 4. Also, I’m not sure it is safe to store the QueryDesc pointer at all
> at the
> ProcessLogQueryPlanInterrupt.
I think the current implementation does not store a pointer to the
queryDesc passed to ProcessLogQueryPlanInterrupt().
That queryDesc is used only to obtain the PlanState nodes that need
to be wrapped. For plan logging itself, LogQueryPlan() uses a
queryDesc obtained separately.
> Maybe the next call to ExecProcNode() will be made
> from an external query (or from a deeper one).
> The current implementation will
> not produce any EXPLAIN in this case, but query still executes.
That said, I think a similar situation can still occur if execution
moves to an external or deeper queryDesc after the individual plan
nodes have been wrapped. In that case, the plan does not be logged
even though the query continues executing.
> I think, combination of a global hook and queryDesc pointer, saved
> somewhere in
> the EState may solve the problem. It is available at each executor's
> node, and
> the EXPLAIN machinery might use an exact descriptor related to the
> currently
> executing query.
Given the performance concern mentioned above, my impression was that
using a global hook would not be a viable approach.
Even if we could address that point, I suspect it would still be
difficult to eliminate this kind of issue entirely, because signal
handling through CFI() and wrapping plan nodes and its execution are
asynchronous.
My understanding is that this problem can occur only when the next
ExecProcNode() call after wrapping the plan nodes happens to be
for an inner or external query.
So personally, I hope it might be acceptable to treat this as a
case where the backend did not reach a point at which the plan
could be logged, as described in the documentation change below:
+ The plan is logged when the target backend next reaches a
+ point where the running plan can be inspected. (..snip..)
+ If the query finishes before such a point is
+ reached, no plan may be logged.
> 5. The phrase ‘only the plan of the most deeply nested query is logged’
> is
> missed in the docs. So the current decision has not yet been
> documented.
Added the explanation:
+ If the target backend is executing a nested query when it
+ processes the request, only the plan of the innermost
+ executing query is logged.
On Thu, Jul 2, 2026 at 8:06 PM Andrei Lepikhov <[email protected]>
wrote:
> The only 'es→signaled' flag is an eyesore. It would be better to base
> the
> decision on the es→analyze flag, which would break the long-standing
> auto_explain rule, if I understand correctly. So, at least this flag
> should be
> renamed to something like es→running, or es→query_in_progress.
Modified to es->running.
> Also, not sure I understand how this code regulates the explain format,
> verbosity and other EXPLAIN settings.
As discussed in the following thread, the current implementation uses
the default
EXPLAIN options and does not provide any particular way to regulate
them:
https://www.postgresql.org/message-id/57ee0cf0cf76b742b0144bd481c56b57%40oss.nttdata.com
As mentioned in that discussion, I think it would be possible to make
this configurable
in the future, for example by adding GUCs or by adding options to
pg_log_query_plan().
For now, however, I would like to focus on developing the mechanism for
obtaining the
plan of a currently running query.
Thanks,
--
Atsushi Torikoshi
Seconded from NTT DATA CORPORATION to SRA OSS K.K.
Attachments:
[text/x-diff] v58-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch (38.2K, ../../[email protected]/2-v58-0001-Add-function-to-log-the-plan-of-the-currently-ru.patch)
download | inline diff:
From a6b36695ddda83c4c732c5d60bf8a8e8037abf51 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 6 Jul 2026 14:53:29 +0900
Subject: [PATCH v58] Add function to log the plan of the currently running
query
Currently, we have to wait for the query execution to finish
to know its plan either using EXPLAIN ANALYZE or auto_explain.
This is not so convenient, for example when investigating
long-running queries on production environments.
To improve this situation, this patch adds pg_log_query_plan()
function that requests to log the plan of the currently
executing query.
On receipt of the request, ExecProcNodes of the current plan
node and its subsidiary nodes are wrapped with
ExecProcNodeFirst, which implements logging query plan.
When the executor next invokes one of the wrapped nodes, the
query plan is logged.
Our initial idea was to send a signal to the target backend
process, which invokes EXPLAIN logic at the next
CHECK_FOR_INTERRUPTS() call. However, we realized during
prototyping that EXPLAIN is complex and may not be safely
executed at arbitrary interrupt points.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
Todo: Consider removing the PID from the log output of
pg_log_backend_memory_contexts() and pg_log_query_plan().
For detail, see the discussion:
https://www.postgresql.org/message-id/CA%2BTgmoY81r7npTS34N_5MLA_u6ghfor5HoSaar53veXUYu1OxQ%40mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 29 +--
doc/src/sgml/func/func-admin.sgml | 37 ++++
src/backend/access/transam/xact.c | 33 ++++
src/backend/commands/Makefile | 1 +
src/backend/commands/explain.c | 43 +++-
src/backend/commands/explain_running.c | 187 ++++++++++++++++++
src/backend/commands/meson.build | 1 +
src/backend/executor/execMain.c | 17 ++
src/backend/executor/execProcnode.c | 132 ++++++++++++-
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/access/xact.h | 3 +
src/include/catalog/pg_proc.dat | 7 +
src/include/commands/explain.h | 3 +
src/include/commands/explain_running.h | 24 +++
src/include/commands/explain_state.h | 1 +
src/include/executor/executor.h | 1 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 2 +
src/test/modules/test_misc/meson.build | 1 +
.../test_misc/t/015_pg_log_query_plan.pl | 101 ++++++++++
src/test/regress/expected/misc_functions.out | 38 ++++
src/test/regress/sql/misc_functions.sql | 31 +++
24 files changed, 666 insertions(+), 37 deletions(-)
create mode 100644 src/backend/commands/explain_running.c
create mode 100644 src/include/commands/explain_running.h
create mode 100644 src/test/modules/test_misc/t/015_pg_log_query_plan.pl
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 97ce498d0c1..07014cab366 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -451,32 +451,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
apply_extension_options(es, extension_options);
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- if (explain_per_plan_hook)
- (*explain_per_plan_hook) (queryDesc->plannedstmt,
- NULL, es,
- queryDesc->sourceText,
- queryDesc->params,
- queryDesc->estate->es_queryEnv);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainStringAssemble(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml
index 0eae1c1f616..15e68edc41b 100644
--- a/doc/src/sgml/func/func-admin.sgml
+++ b/doc/src/sgml/func/func-admin.sgml
@@ -184,6 +184,43 @@
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para>
+ <para>
+ The plan is logged when the target backend next reaches a
+ point where the running plan can be inspected. If the backend
+ is spending a long time in work that does not reach such a
+ point, the log output can be delayed until that work
+ completes. If the query finishes before such a point is
+ reached, no plan may be logged.
+ </para>
+ <para>
+ If the target backend is executing a nested query when it
+ processes the request, only the plan of the innermost
+ executing query is logged.
+ </para>
+ <para>
+ This function is restricted to superusers by default, but other
+ users can be granted EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..404898b97ab 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,7 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ QueryDesc *queryDesc; /* my current QueryDesc */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -250,6 +251,7 @@ static TransactionStateData TopTransactionStateData = {
.state = TRANS_DEFAULT,
.blockState = TBLOCK_DEFAULT,
.topXidLogged = false,
+ .queryDesc = NULL,
};
/*
@@ -935,6 +937,27 @@ GetCurrentTransactionNestLevel(void)
return s->nestingLevel;
}
+/*
+ * SetCurrentQueryDesc
+ */
+void
+SetCurrentQueryDesc(QueryDesc *queryDesc)
+{
+ TransactionState s = CurrentTransactionState;
+
+ s->queryDesc = queryDesc;
+}
+
+/*
+ * GetCurrentQueryDesc
+ */
+QueryDesc *
+GetCurrentQueryDesc(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->queryDesc;
+}
/*
* TransactionIdIsCurrentTransactionId
@@ -2955,6 +2978,9 @@ AbortTransaction(void)
/* Reset snapshot export state. */
SnapBuildResetExportedSnapshotState();
+ /* Reset current QueryDesc. */
+ SetCurrentQueryDesc(NULL);
+
/*
* If this xact has started any unfinished parallel operation, clean up
* its workers and exit parallel mode. Don't warn about leaked resources.
@@ -5355,6 +5381,13 @@ AbortSubTransaction(void)
/* Reset logical streaming state. */
ResetLogicalStreamingState();
+ /*
+ * Reset current QueryDesc. Note that even after this reset, it's still
+ * possible to obtain the parent transaction's query plans, since they are
+ * preserved in standard_ExecutorRun().
+ */
+ SetCurrentQueryDesc(NULL);
+
/*
* No need for SnapBuildResetExportedSnapshotState() here, snapshot
* exports are not supported in subtransactions.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 5b9d084977e..671f036fa76 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -35,6 +35,7 @@ OBJS = \
explain.o \
explain_dr.o \
explain_format.o \
+ explain_running.o \
explain_state.o \
extension.o \
foreigncmds.o \
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index a40d03d35f3..d2cf3c940a7 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1092,6 +1092,43 @@ ExplainQueryParameters(ExplainState *es, ParamListInfo params, int maxlen)
ExplainPropertyText("Query Parameters", str, es);
}
+/*
+ * ExplainStringAssemble -
+ * Assemble es->str for logging according to specified options and format
+ */
+
+void
+ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ if (explain_per_plan_hook)
+ (*explain_per_plan_hook) (queryDesc->plannedstmt,
+ NULL, es,
+ queryDesc->sourceText,
+ queryDesc->params,
+ queryDesc->estate->es_queryEnv);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* report_triggers -
* report execution stats for a single relation's triggers
@@ -1827,7 +1864,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
- * haven't done ExecutorEnd yet. This is pretty grotty ...
+ * haven't done ExecutorEnd yet. This is pretty grotty ... This cleanup
+ * should not be done when explaining a running query, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1835,7 +1874,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->running)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
diff --git a/src/backend/commands/explain_running.c b/src/backend/commands/explain_running.c
new file mode 100644
index 00000000000..43204dc349f
--- /dev/null
+++ b/src/backend/commands/explain_running.c
@@ -0,0 +1,187 @@
+/*-------------------------------------------------------------------------
+ *
+ * explain_running.c
+ * Explain query plans during execution
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/explain_running.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "commands/explain.h"
+#include "commands/explain_format.h"
+#include "commands/explain_running.h"
+#include "commands/explain_state.h"
+#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "storage/procsignal.h"
+#include "utils/backend_status.h"
+#include "utils/injection_point.h"
+
+/* Is plan node wrapping for query plan logging currently in progress? */
+static bool WrapNodesInProgress = false;
+
+/*
+ * Handle receipt of an interrupt indicating logging the plan of the currently
+ * running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * Actual plan logging function.
+ */
+void
+LogQueryPlan(void)
+{
+ ExplainState *es;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ QueryDesc *queryDesc;
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+ es->running = true;
+
+ /*
+ * Current QueryDesc is valid only during standard_ExecutorRun. However,
+ * ExecProcNode can be called afterward (i.e., ExecPostprocessPlan). To
+ * handle the case, check whether we have QueryDesc now.
+ */
+ queryDesc = GetCurrentQueryDesc();
+
+ PG_TRY();
+ {
+ if (queryDesc != NULL)
+ {
+ ExplainStringAssemble(es, queryDesc, es->format, false, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query and its plan running on backend with PID %d are:\n%s",
+ MyProcPid, es->str->data));
+ }
+ }
+ PG_FINALLY();
+ {
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+ LogQueryPlanPending = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Process the request for logging query plan at CHECK_FOR_INTERRUPTS().
+ *
+ * Since executing EXPLAIN-related code at an arbitrary CHECK_FOR_INTERRUPTS()
+ * point is potentially unsafe, this function just wraps the nodes of
+ * ExecProcNode with ExecProcNodeFirst, which logs query plan if requested.
+ * This way ensures that EXPLAIN-related code is executed only during
+ * ExecProcNodeFirst, where it is considered safe.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ QueryDesc *querydesc = GetCurrentQueryDesc();
+
+ /* If current query has already finished, we can do nothing but exit */
+ if (querydesc == NULL)
+ {
+ LogQueryPlanPending = false;
+ return;
+ }
+
+ /*
+ * Exit immediately if wrapping plan is already in progress. This prevents
+ * recursive calls, which could occur if logging is requested repeatedly
+ * and rapidly, potentially leading to infinite recursion and crash.
+ */
+ if (WrapNodesInProgress)
+ return;
+
+ WrapNodesInProgress = true;
+
+ PG_TRY();
+ {
+ INJECTION_POINT("log-query-interrupt", NULL);
+
+ /*
+ * Wrap ExecProcNodes with ExecProcNodeFirst, which logs query plan
+ * when LogQueryPlanPending is true.
+ */
+ ExecSetExecProcNodeRecurse(querydesc->planstate);
+ }
+ PG_FINALLY();
+ {
+ WrapNodesInProgress = false;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal a backend to log its
+ * plan because the output is written to the server log, which in many cases
+ * can only be read by superusers.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status = NULL;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc != NULL)
+ be_status = pgstat_get_beentry_by_proc_number(proc->vxid.procNumber);
+
+ if (proc == NULL || be_status == NULL || be_status->st_procpid != pid)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->vxid.procNumber) < 0)
+ {
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index 9f258d566eb..d630dedd5d7 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -23,6 +23,7 @@ backend_sources += files(
'explain.c',
'explain_dr.c',
'explain_format.c',
+ 'explain_running.c',
'explain_state.c',
'extension.c',
'foreigncmds.c',
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index fde502efd38..dc4177216cf 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -44,6 +44,7 @@
#include "access/xact.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
+#include "commands/explain_running.h"
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/executor.h"
@@ -323,6 +324,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
DestReceiver *dest;
bool sendTuples;
MemoryContext oldcontext;
+ QueryDesc *oldQueryDesc;
/* sanity checks */
Assert(queryDesc != NULL);
@@ -335,6 +337,13 @@ standard_ExecutorRun(QueryDesc *queryDesc,
/* caller must ensure the query's snapshot is active */
Assert(GetActiveSnapshot() == estate->es_snapshot);
+ /*
+ * Save current QueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ oldQueryDesc = GetCurrentQueryDesc();
+ SetCurrentQueryDesc(queryDesc);
+
/*
* Switch into per-query memory context
*/
@@ -397,6 +406,14 @@ standard_ExecutorRun(QueryDesc *queryDesc,
InstrStop(queryDesc->query_instr);
MemoryContextSwitchTo(oldcontext);
+ SetCurrentQueryDesc(oldQueryDesc);
+
+ /*
+ * Ensure LogQueryPlanPending is initialized in case there was no time for
+ * logging the plan. Otherwise plan will be logged at the next query
+ * execution on the same session.
+ */
+ LogQueryPlanPending = false;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 7c4c66e323f..b8c118dea4f 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -72,6 +72,7 @@
*/
#include "postgres.h"
+#include "commands/explain_running.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeAgg.h"
@@ -431,9 +432,10 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
/*
* Add a wrapper around the ExecProcNode callback that checks stack depth
- * during the first execution and maybe adds an instrumentation wrapper.
- * When the callback is changed after execution has already begun that
- * means we'll superfluously execute ExecProcNodeFirst, but that seems ok.
+ * and query logging request during the execution and maybe adds an
+ * instrumentation wrapper. When the callback is changed after execution
+ * has already begun that means we'll superfluously execute
+ * ExecProcNodeFirst, but that seems ok.
*/
node->ExecProcNodeReal = function;
node->ExecProcNode = ExecProcNodeFirst;
@@ -441,21 +443,37 @@ ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
/*
- * ExecProcNode wrapper that performs some one-time checks, before calling
+ * ExecProcNode wrapper that performs some extra checks, before calling
* the relevant node method (possibly via an instrumentation wrapper).
+ *
+ * Normally, this is just invoked once for the first call to any given node,
+ * and thereafter we arrange to call ExecProcNodeInstr or the relevant node
+ * method directly. However, it's legal to reset node->ExecProcNode back to
+ * this function at any time, and we do that whenever the query plan might
+ * need to be printed, so that we only incur the cost of checking for that
+ * case when required.
*/
static TupleTableSlot *
ExecProcNodeFirst(PlanState *node)
{
/*
- * Perform stack depth check during the first execution of the node. We
- * only do so the first time round because it turns out to not be cheap on
- * some common architectures (eg. x86). This relies on the assumption
- * that ExecProcNode calls for a given plan node will always be made at
- * roughly the same stack depth.
+ * Perform a stack depth check. We don't want to do this all the time
+ * because it turns out to not be cheap on some common architectures (eg.
+ * x86). This relies on the assumption that ExecProcNode calls for a
+ * given plan node will always be made at roughly the same stack depth.
*/
check_stack_depth();
+ /*
+ * If we have been asked to print the query plan, do that now. We dare not
+ * try to do this directly from CHECK_FOR_INTERRUPTS() because we don't
+ * really know what the executor state is at that point, but we assume
+ * that when entering a node the state will be sufficiently consistent
+ * that trying to print the plan makes sense.
+ */
+ if (LogQueryPlanPending)
+ LogQueryPlan();
+
/*
* If instrumentation is required, change the wrapper to one that just
* does instrumentation. Otherwise we can dispense with all wrappers and
@@ -527,6 +545,102 @@ MultiExecProcNode(PlanState *node)
return result;
}
+/*
+ * Wrap array of PlanState ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+ExecSetExecProcNodeArray(PlanState **planstates, int nplans)
+{
+ int i;
+
+ for (i = 0; i < nplans; i++)
+ ExecSetExecProcNodeRecurse(planstates[i]);
+}
+
+/*
+ * Wrap CustomScanState children's ExecProcNode with ExecProcNodeFirst.
+ */
+static void
+CSSChildExecSetExecProcNodeArray(CustomScanState *css)
+{
+ ListCell *cell;
+
+ foreach(cell, css->custom_ps)
+ ExecSetExecProcNodeRecurse((PlanState *) lfirst(cell));
+}
+
+/*
+ * Recursively wrap all the underlying ExecProcNode with ExecProcNodeFirst.
+ *
+ * Recursion is usually necessary because the next ExecProcNode() call may be
+ * invoked not only through the current node, but also via lefttree, righttree,
+ * initPlan, subPlan, or other special child plans.
+ */
+void
+ExecSetExecProcNodeRecurse(PlanState *ps)
+{
+ ExecSetExecProcNode(ps, ps->ExecProcNodeReal);
+
+ if (ps->lefttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->lefttree);
+ if (ps->righttree != NULL)
+ ExecSetExecProcNodeRecurse(ps->righttree);
+ if (ps->initPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->initPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+ if (ps->subPlan != NULL)
+ {
+ ListCell *l;
+
+ foreach(l, ps->subPlan)
+ {
+ SubPlanState *sstate = (SubPlanState *) lfirst(l);
+
+ ExecSetExecProcNodeRecurse(sstate->planstate);
+ }
+ }
+
+ /* special child plans */
+ switch (nodeTag(ps->plan))
+ {
+ case T_Append:
+ ExecSetExecProcNodeArray(((AppendState *) ps)->appendplans,
+ ((AppendState *) ps)->as_nplans);
+ break;
+ case T_MergeAppend:
+ ExecSetExecProcNodeArray(((MergeAppendState *) ps)->mergeplans,
+ ((MergeAppendState *) ps)->ms_nplans);
+ break;
+ case T_BitmapAnd:
+ ExecSetExecProcNodeArray(((BitmapAndState *) ps)->bitmapplans,
+ ((BitmapAndState *) ps)->nplans);
+ break;
+ case T_BitmapOr:
+ ExecSetExecProcNodeArray(((BitmapOrState *) ps)->bitmapplans,
+ ((BitmapOrState *) ps)->nplans);
+ break;
+ case T_SubqueryScan:
+ ExecSetExecProcNodeRecurse(((SubqueryScanState *) ps)->subplan);
+ break;
+ case T_CteScan:
+ ExecSetExecProcNodeRecurse(((CteScanState *) ps)->cteplanstate);
+ break;
+ case T_CustomScan:
+ CSSChildExecSetExecProcNodeArray((CustomScanState *) ps);
+ break;
+ default:
+ break;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecEndNode
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 1397f65f67b..4b442d4cdad 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/explain_running.h"
#include "commands/repack.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -713,6 +714,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index ce18df820cd..0e4483def5d 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/explain_running.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
#include "commands/repack.h"
@@ -3704,6 +3705,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index bbd28d14d99..de84fd9cfa3 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -40,6 +40,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/access/xact.h b/src/include/access/xact.h
index a8cbdf247c8..0570fdd3277 100644
--- a/src/include/access/xact.h
+++ b/src/include/access/xact.h
@@ -432,6 +432,7 @@ typedef struct xl_xact_parsed_abort
TimestampTz origin_timestamp;
} xl_xact_parsed_abort;
+typedef struct QueryDesc QueryDesc;
/* ----------------
* extern definitions
@@ -458,6 +459,8 @@ extern TimestampTz GetCurrentStatementStartTimestamp(void);
extern TimestampTz GetCurrentTransactionStopTimestamp(void);
extern void SetCurrentStatementStartTimestamp(void);
extern int GetCurrentTransactionNestLevel(void);
+extern QueryDesc *GetCurrentQueryDesc(void);
+extern void SetCurrentQueryDesc(QueryDesc *queryDesc);
extern bool TransactionIdIsCurrentTransactionId(TransactionId xid);
extern int GetTopReadOnlyTransactionNestLevel(void);
extern void CommandCounterIncrement(void);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..741f04191bb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8740,6 +8740,13 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts', proacl => '{POSTGRES=X}' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan',
+ proacl => '{POSTGRES=X}' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 472e141bba3..633ac94ea58 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -81,5 +81,8 @@ extern void ExplainPrintJITSummary(ExplainState *es,
extern void ExplainQueryText(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainQueryParameters(ExplainState *es,
ParamListInfo params, int maxlen);
+extern void ExplainStringAssemble(ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
#endif /* EXPLAIN_H */
diff --git a/src/include/commands/explain_running.h b/src/include/commands/explain_running.h
new file mode 100644
index 00000000000..316ef25be60
--- /dev/null
+++ b/src/include/commands/explain_running.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * explain_running.h
+ * prototypes for explain_running.c
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/explain_running.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef EXPLAIN_RUNNING_H
+#define EXPLAIN_RUNNING_H
+
+#include "executor/executor.h"
+#include "commands/explain_state.h"
+
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
+extern void LogQueryPlan(void);
+extern Datum pg_log_query_plan(PG_FUNCTION_ARGS);
+
+#endif /* EXPLAIN_RUNNING_H */
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 97bc7ed49f6..4b90f5eaa0a 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -73,6 +73,7 @@ typedef struct ExplainState
* entry */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool running; /* whether target query is running */
/* extensions */
void **extension_state;
int extension_state_allocated;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 1798e6027d4..7e99fb2afed 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -299,6 +299,7 @@ extern void EvalPlanQualEnd(EPQState *epqstate);
extern PlanState *ExecInitNode(Plan *node, EState *estate, int eflags);
extern void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function);
extern Node *MultiExecProcNode(PlanState *node);
+extern void ExecSetExecProcNodeRecurse(PlanState *ps);
extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 7170a4bff98..0ab775e964f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -98,6 +98,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index aaa158bfd66..c2142043e2d 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,8 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
+ * query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
PROCSIG_SLOTSYNC_MESSAGE, /* ask slot synchronization to stop */
PROCSIG_REPACK_MESSAGE, /* Message from repack worker */
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index ee290698b31..8e8c8658120 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -23,6 +23,7 @@ tests += {
't/012_ddlutils.pl',
't/013_temp_obj_multisession.pl',
't/014_log_statement_max_length.pl',
+ 't/015_pg_log_query_plan.pl',
],
# The injection points are cluster-wide, so disable installcheck
'runningcheck': false,
diff --git a/src/test/modules/test_misc/t/015_pg_log_query_plan.pl b/src/test/modules/test_misc/t/015_pg_log_query_plan.pl
new file mode 100644
index 00000000000..2055641283e
--- /dev/null
+++ b/src/test/modules/test_misc/t/015_pg_log_query_plan.pl
@@ -0,0 +1,101 @@
+
+# Copyright (c) 2024-2026, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+use locale;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test that pg_log_query_plan() actually logs the query plan of
+# another backend executing a query.
+
+# This test requires timing coordinations:
+# 1) The target backend must be executing a query when
+# pg_log_query_plan() sends the signal.
+# 2) We must confirm that the target backend actually received the
+# signal that requests logging of the plan.
+#
+# We use an advisory lock and an injection point to control them
+# respectively.
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available, as it may be
+# possible that this script is run with installcheck, where the module
+# would not be installed by default.
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+my $psql_session1 = $node->background_psql('postgres');
+my $psql_session2 = $node->background_psql('postgres');
+
+my $session1_pid = $psql_session1->query_safe("select pg_backend_pid()");
+
+# Set injection point in the logging plan request handler to ensure
+# that session1 received the signal of pg_log_query_plan().
+$psql_session1->query_safe(
+ qq[
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('log-query-interrupt', 'wait');
+]);
+
+# Use an advisory lock to make session1 blocked during query execution.
+$psql_session2->query_safe(
+ qq[
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+]);
+
+$psql_session1->query_until(
+ qr/wait_on_advisory_lock/, q(
+ \echo wait_on_advisory_lock
+ BEGIN;
+ SELECT pg_advisory_xact_lock(1);
+));
+
+# Confirm that session1 is actually waiting on the advisory lock.
+$node->wait_for_event('client backend', 'advisory');
+
+# Run pg_log_query_plan().
+$psql_session2->query_safe("SELECT pg_log_query_plan($session1_pid);");
+
+# Ensure that the signal of pg_log_query_plan() is actually
+# received by confirming session1 is waiting on the injection point.
+$node->wait_for_event('client backend', 'log-query-interrupt');
+
+# Commit the session 2 to release the advisory lock.
+$psql_session2->query_safe("COMMIT;");
+
+my $log_offset = -s $node->logfile;
+
+# Detach the injection point to start logging the plan.
+$psql_session2->query_safe(
+ qq[
+ SELECT injection_points_wakeup('log-query-interrupt');
+ SELECT injection_points_detach('log-query-interrupt');
+]);
+
+$node->wait_for_log('query and its plan running on backend with PID',
+ $log_offset);
+
+$psql_session1->query_safe("COMMIT;");
+
+ok($psql_session1->quit);
+ok($psql_session2->quit);
+
+done_testing();
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c3261bff209..cc1f5103fef 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -357,6 +357,44 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
FROM regress_log_memory;
DROP ROLE regress_log_memory;
--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/015_pg_log_query_plan.pl.
+CREATE ROLE regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+DROP ROLE regress_log_plan;
+--
-- Test some built-in SRFs
--
-- The outputs of these are variable, so we can't just print their results
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 946ee5726cd..9c508175575 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -113,6 +113,37 @@ REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
DROP ROLE regress_log_memory;
+--
+-- pg_log_query_plan()
+--
+-- Plans are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the optimizer.
+-- Here we just verifies that the permissions are set properly.
+--
+-- The test that verifies the backend's query plan is actually
+-- logged is implemented in
+-- src/test/modules/test_misc/t/015_pg_log_query_plan.pl.
+
+CREATE ROLE regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_plan;
+
+SELECT has_function_privilege('regress_log_plan',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_plan;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_plan;
+
+DROP ROLE regress_log_plan;
+
--
-- Test some built-in SRFs
--
--
2.48.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-01 10:42 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-01 13:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-02 11:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-06 06:21 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2026-07-06 13:05 ` Andrei Lepikhov <[email protected]>
2026-07-07 14:46 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Andrei Lepikhov @ 2026-07-06 13:05 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Robert Haas <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On 06/07/2026 08:21, torikoshia wrote:
> On Wed, Jul 1, 2026 at 7:42 PM Andrei Lepikhov <[email protected]> wrote:
>
>> I’d
>> propose introducing a tiny hook (register callback?) into ExecProcNode() and
>> just setting it in ProcessLogQueryPlanInterrupt.
>
> This approach would also be possible, but we are concerned that
> checking the hook on every ExecProcNode() call would add too much
> overhead and could have a significant performance impact.
I doubt the performance impact, but generally agree: having
ExecSetExecProcNode/ExecProcNodeFirst trick with a one-time callback inside
(right now it is the only hard-wired LogQueryPlan call) may be enough for the
implementation.
So, the extendable part here may be an ExecProcNodeFirst callback that an
extension module can register once (for example, before the start of an EXPLAIN
ANALYZE) or re-arm on each call of such 'execution state probes'.
>> Maybe the next call to ExecProcNode() will be made
>> from an external query (or from a deeper one).
>> The current implementation will
>> not produce any EXPLAIN in this case, but query still executes.
>
> That said, I think a similar situation can still occur if execution
> moves to an external or deeper queryDesc after the individual plan
> nodes have been wrapped. In that case, the plan does not be logged
> even though the query continues executing.
That's the problem. It makes tests unstable as well as user experience
unpredictable. How to script (automatise) this feature if you have no
guarantees? Is anywhere in Postgres any other examples of such behaviour? To
close the gap, ExecutorStart routine could check a global 'pending report' flag
and wrap planstate nodes if needed.
Also, there are kinds of query feedback that might be, reporting the ‘pending
reports’ like queryid to the pg_stat_activity. That solves the automation
problem for the async feature.
In addition, I wonder why this code doesn't use standard planstate walker. It
seems to me that something like the following should work:
static bool
rearm_execprocnode_walker(PlanState *node, void *context)
{
if (node == NULL)
return false;
ExecSetExecProcNode(node, node->ExecProcNodeReal);
return planstate_tree_walker(node, rearm_execprocnode_walker, context);
}
/* trigger point */
(void) rearm_execprocnode_walker(queryDesc->planstate, NULL);
--
regards, Andrei Lepikhov,
pgEdge
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-01 10:42 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-01 13:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-02 11:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-06 06:21 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-06 13:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
@ 2026-07-07 14:46 ` torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2026-07-07 14:46 UTC (permalink / raw)
To: Andrei Lepikhov <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Robert Haas <[email protected]>; [email protected]; Atsushi Torikoshi <[email protected]>; [email protected]; [email protected]
On 2026-07-06 22:05, Andrei Lepikhov wrote:
> On 06/07/2026 08:21, torikoshia wrote:
>> On Wed, Jul 1, 2026 at 7:42 PM Andrei Lepikhov <[email protected]>
>> wrote:
>>
>>> I’d
>>> propose introducing a tiny hook (register callback?) into
>>> ExecProcNode() and
>>> just setting it in ProcessLogQueryPlanInterrupt.
>>
>> This approach would also be possible, but we are concerned that
>> checking the hook on every ExecProcNode() call would add too much
>> overhead and could have a significant performance impact.
>
> I doubt the performance impact, but generally agree: having
> ExecSetExecProcNode/ExecProcNodeFirst trick with a one-time callback
> inside
> (right now it is the only hard-wired LogQueryPlan call) may be enough
> for the
> implementation.
>
> So, the extendable part here may be an ExecProcNodeFirst callback that
> an
> extension module can register once (for example, before the start of an
> EXPLAIN
> ANALYZE) or re-arm on each call of such 'execution state probes'.
>
>>> Maybe the next call to ExecProcNode() will be made
>>> from an external query (or from a deeper one).
>>> The current implementation will
>>> not produce any EXPLAIN in this case, but query still executes.
>>
>> That said, I think a similar situation can still occur if execution
>> moves to an external or deeper queryDesc after the individual plan
>> nodes have been wrapped. In that case, the plan does not be logged
>> even though the query continues executing.
>
> That's the problem.
Sorry, I misremembered the logic.
In this case, even if execution moves to an external or deeper
QueryDesc, LogQueryPlanPending flag remains set. Therefore, the plan
tree for that external or deeper QueryDesc will be wrapped, and
the plan will be logged.
For example, with the following query, ExecProcNode() is called four
times:
CREATE OR REPLACE FUNCTION f1() RETURNS void AS $$
BEGIN
PERFORM 1;
END;
$$ LANGUAGE plpgsql;
SELECT f1();
If pg_log_query_plan() is called just after the first
ExecProcNode() call reaches ExecProcNodeFirst(), the plan for PERFORM 1
is logged.
Thanks,
--
Atsushi Torikoshi
Seconded from NTT DATA CORPORATION to SRA OSS K.K.
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
@ 2022-02-08 08:18 Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Kyotaro Horiguchi @ 2022-02-08 08:18 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]
At Tue, 8 Feb 2022 01:13:44 +0900, Fujii Masao <[email protected]> wrote in
>
>
> On 2022/02/02 21:59, torikoshia wrote:
> >> This may cause users to misunderstand that pg_log_query_plan() fails
> >> while the target backend is holding *any* locks? Isn't it better to
> >> mention "page-level locks", instead? So how about the following?
> >>
> >> --------------------------
> >> Note that the request to log the query plan will be ignored if it's
> >> received during a short period while the target backend is holding a
> >> page-level lock.
> >> --------------------------
> > Agreed.
>
> On second thought, this note is confusing rather than helpful? Because
> the users don't know when and what operation needs page-level lock. So
> now I'm thinking it's better to remove this note.
*I* agree to removing the note. And the following error message looks
as mysterious as the note is, and the DETAIL doesn't help..
ereport(LOG_SERVER_ONLY,
+ errmsg("could not log the query plan"),
+ errdetail("Cannot log the query plan while holding page-level lock."));
+ hash_seq_term(&status);
We should tell the command can be retried soon, like this?
"LOG: ignored request for logging query plan due to lock confilcts"
"HINT: You can try again in a moment."
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
@ 2022-02-08 15:12 ` torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2022-02-08 15:12 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; [email protected]; +Cc: [email protected]
On 2022-02-03 17:09, Robert Treat wrote:
> On Wed, Feb 2, 2022 at 7:59 AM torikoshia <[email protected]>
> wrote:
>>
>> 2022-02-01 01:51, Fujii Masao wrote:
> <snip>
>> > + Note that nested statements (statements executed inside a
>> > function) are not
>> > + considered for logging. Only the plan of the most deeply nested
>> > query is logged.
>> >
>> > Now the plan of even nested statement can be logged. So this
>> > description needs to be updated?
>>
>> Modified it as below:
>>
>> - Note that nested statements (statements executed inside a
>> function) are not
>> - considered for logging. Only the plan of the most deeply
>> nested
>> query is logged.
>> + Note that when the statements are executed inside a
>> function,
>> only the
>> + plan of the most deeply nested query is logged.
>>
>
> Minor nit, but I think the "the" is superfluous.. ie.
>
> "Note that when statements are executed inside a function,
> only the plan of the most deeply nested query is logged"
Thanks! Modified it.
On 2022-02-08 01:13, Fujii Masao wrote:
Thanks for the comments!
> On 2022/02/02 21:59, torikoshia wrote:
>>> This may cause users to misunderstand that pg_log_query_plan() fails
>>> while the target backend is holding *any* locks? Isn't it better to
>>> mention "page-level locks", instead? So how about the following?
>>>
>>> --------------------------
>>> Note that the request to log the query plan will be ignored if it's
>>> received during a short period while the target backend is holding a
>>> page-level lock.
>>> --------------------------
>>
>> Agreed.
>
> On second thought, this note is confusing rather than helpful? Because
> the users don't know when and what operation needs page-level lock. So
> now I'm thinking it's better to remove this note.
Removed it.
>
>
>> As you pointed out offlist, the issue could occur even when both
>> pg_log_backend_memory_contexts and pg_log_query_plan are called and it
>> may be better to make another patch.
>
> OK.
>
>
>> You also pointed out offlist that it would be necessary to call
>> LockErrorCleanup() if ProcessLogQueryPlanInterrupt() can incur ERROR.
>> AFAICS it can call ereport(ERROR), i.e., palloc0() in
>> NewExplainState(), so I added PG_TRY/CATCH and make it call
>> LockErrorCleanup() when ERROR occurs.
>
> As we discussed off-list, this treatment might not be necessary.
> Because when ERROR or FATAL error happens, AbortTransaction() is
> called and LockErrorCleanup() is also called inside there.
Agreed.
>> Since I'm not sure how long it will take to discuss this point, the
>> attached patch is based on the current HEAD at this time.
>
> Thanks for updating the patch!
>
> @@ -5048,6 +5055,12 @@ AbortSubTransaction(void)
> */
> PG_SETMASK(&UnBlockSig);
> + /*
> + * When ActiveQueryDesc is referenced after abort, some of its
> elements
> + * are freed. To avoid accessing them, reset ActiveQueryDesc here.
> + */
> + ActiveQueryDesc = NULL;
>
> AbortSubTransaction() should reset ActiveQueryDesc to
> save_ActiveQueryDesc that ExecutorRun() set, instead of NULL?
> Otherwise ActiveQueryDesc of top-level statement will be unavailable
> after subtransaction is aborted in the nested statements.
>
> For example, no plan is logged while the following "select
> pg_sleep(test())" is running, because the exception inside test()
> function aborts the subtransaction and resets ActiveQueryDesc to NULL.
>
> create or replace function test () returns int as $$
> begin
> perform 1/0;
> exception when others then
> return 30;
> end;
> $$ language plpgsql;
>
> select pg_sleep(test());
Agreed.
BTW, since the above example results in calling ExecutorRun() only once,
the output didn't differ even after ActiveQueryDesc is reset to
save_ActiveQueryDesc.
The below definition of test() worked as expected.
create or replace function test () returns int as $$
begin
perform 1;
perform 1/0;
exception when others then
return 30;
end;
$$ language plpgsql;
>
>
> -CREATE ROLE regress_log_memory;
> +CREATE ROLE regress_log;
>
> Isn't this name too generic? How about regress_log_function, for
> example?
Agreed.
On 2022-02-08 17:18, Kyotaro Horiguchi wrote:
> At Tue, 8 Feb 2022 01:13:44 +0900, Fujii Masao
> <[email protected]> wrote in
>>
>>
>> On 2022/02/02 21:59, torikoshia wrote:
>> >> This may cause users to misunderstand that pg_log_query_plan() fails
>> >> while the target backend is holding *any* locks? Isn't it better to
>> >> mention "page-level locks", instead? So how about the following?
>> >>
>> >> --------------------------
>> >> Note that the request to log the query plan will be ignored if it's
>> >> received during a short period while the target backend is holding a
>> >> page-level lock.
>> >> --------------------------
>> > Agreed.
>>
>> On second thought, this note is confusing rather than helpful? Because
>> the users don't know when and what operation needs page-level lock. So
>> now I'm thinking it's better to remove this note.
>
> *I* agree to removing the note. And the following error message looks
> as mysterious as the note is, and the DETAIL doesn't help..
>
> ereport(LOG_SERVER_ONLY,
> + errmsg("could not log the query plan"),
> + errdetail("Cannot log the query plan while holding page-level
> lock."));
> + hash_seq_term(&status);
>
> We should tell the command can be retried soon, like this?
>
> "LOG: ignored request for logging query plan due to lock confilcts"
> "HINT: You can try again in a moment."
Thanks, it seems better.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v20-0001-log-running-query-plan.patch (24.9K, ../../[email protected]/2-v20-0001-log-running-query-plan.patch)
download | inline diff:
From 1df46b95a26d85835fb05b13b01dd502f8fc24cd Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 8 Feb 2022 23:26:16 +0900
Subject: [PATCH v20] Add function to log the plan of the query
currently running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat
---
doc/src/sgml/func.sgml | 44 ++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 140 ++++++++++++++++++-
src/backend/executor/execMain.c | 9 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 ++++--
17 files changed, 308 insertions(+), 28 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8754f2f89b..a3c2fd4db4 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25461,6 +25461,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25574,6 +25593,31 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index c9516e03fa..d5e5ced48c 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -59,6 +59,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2723,6 +2724,12 @@ AbortTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5048,6 +5055,12 @@ AbortSubTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = save_ActiveQueryDesc;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index fd1421788e..b90a8a0537 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -711,6 +711,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index b970997c34..2f5aa86442 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +43,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1594,6 +1596,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by singal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1601,7 +1606,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4925,3 +4930,134 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 549d9eb696..aeeff85698 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,10 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+QueryDesc *save_ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +303,15 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index f1c8ff8f9e..82691193b1 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -661,6 +662,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 5f5803f681..9beb456f78 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index fda2e9360e..d6a51be565 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "executor/spi.h"
@@ -3367,6 +3368,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index c26a1a73df..116cd69699 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7024dbe10a..bb4e12ef62 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8047,6 +8047,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4550', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 666977fb1f..a59abbf53d 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 02276d3edd..e8427b9655 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index dc537e20f2..6a4150a3cf 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index a121e65066..00fe169035 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index f9a6882ecb..227d24b9d6 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,8 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
+extern PGDLLIMPORT QueryDesc *save_ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 6cf39c6bd2..1df296c075 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -268,14 +268,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -289,8 +290,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -298,15 +299,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -315,8 +316,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index cfaba456e1..eef1a16ab0 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -57,39 +57,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: ba15f16107bea8a93edc505f3013cd7df4ac90fc
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2022-02-21 17:39 ` Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Fujii Masao @ 2022-02-21 17:39 UTC (permalink / raw)
To: torikoshia <[email protected]>; Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]
On 2022/02/09 0:12, torikoshia wrote:
> BTW, since the above example results in calling ExecutorRun() only once, the output didn't differ even after ActiveQueryDesc is reset to save_ActiveQueryDesc.
>
> The below definition of test() worked as expected.
>
> create or replace function test () returns int as $$
> begin
> perform 1;
> perform 1/0;
> exception when others then
> return 30;
> end;
> $$ language plpgsql;
So in this case ActiveQueryDesc set by ExecutorRun() called only once is still valid even when AbortSubTransaction() is called. That is, that ActiveQueryDesc should NOT be reset to save_ActiveQueryDesc in this case, should it?
OTOH, in your example, ActiveQueryDesc set by the second call to ExecutorRun() should be reset in AbortSubTransaction(). Then ActiveQueryDesc set by the first call should be valid.
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
@ 2022-03-09 10:04 ` torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2022-03-09 10:04 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: [email protected]
On 2022-02-08 01:13, Fujii Masao wrote:
> AbortSubTransaction() should reset ActiveQueryDesc to
> save_ActiveQueryDesc that ExecutorRun() set, instead of NULL?
> Otherwise ActiveQueryDesc of top-level statement will be unavailable
> after subtransaction is aborted in the nested statements.
I once agreed above suggestion and made v20 patch making
save_ActiveQueryDesc a global variable, but it caused segfault when
calling pg_log_query_plan() after FreeQueryDesc().
OTOH, doing some kind of reset of ActiveQueryDesc seems necessary since
it also caused segfault when running pg_log_query_plan() during
installcheck.
There may be a better way, but resetting ActiveQueryDesc to NULL seems
safe and simple.
Of course it makes pg_log_query_plan() useless after a subtransaction is
aborted.
However, if it does not often happen that people want to know the
running query's plan whose subtransaction is aborted, resetting
ActiveQueryDesc to NULL would be acceptable.
Attached is a patch that sets ActiveQueryDesc to NULL when a
subtransaction is aborted.
How do you think?
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v21-0001-log-running-query-plan.patch (25.1K, ../../[email protected]/2-v21-0001-log-running-query-plan.patch)
download | inline diff:
From 5be784278e8e7aeeeadf60a772afccda7b59e6e4 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 9 Mar 2022 18:18:06 +0900
Subject: [PATCH v21] Add function to log the plan of the query currently
running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat
---
doc/src/sgml/func.sgml | 49 +++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 140 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 1 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 ++++--
17 files changed, 314 insertions(+), 28 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8a802fb225..0722225056 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25461,6 +25461,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25574,6 +25593,36 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 8964ddf3eb..19e4933edd 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -60,6 +60,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2724,6 +2725,12 @@ AbortTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5045,6 +5052,12 @@ AbortSubTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 81bac6f581..512a344b42 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -709,6 +709,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index de81379da3..60120c2067 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +43,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1600,6 +1602,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by singal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1607,7 +1612,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -4931,3 +4936,134 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 549d9eb696..fe9b6a9229 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -76,6 +76,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -299,10 +302,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index f41563a0a4..7208dc3a68 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -652,6 +653,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index ee2e15c17e..bb6dde402b 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index d7e39aed64..c8a9e52896 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3398,6 +3399,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 3419c099b2..1e055884f3 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d8e8715ed1..aefb1e5182 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8053,6 +8053,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4550', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 666977fb1f..a59abbf53d 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0abc3ad540..9eaf309159 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index dc537e20f2..6a4150a3cf 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index ee636900f3..dc8fd66466 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index f9a6882ecb..227d24b9d6 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,8 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
+extern PGDLLIMPORT QueryDesc *save_ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 8567fcc2b4..c9b8168dca 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 3db3f8bade..911b0d37c4 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: 7687ca996e558d95e68d2d0d70fed22a6317ba78
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2022-05-16 08:02 ` torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2022-05-16 08:02 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; [email protected]
On 2022-03-09 19:04, torikoshia wrote:
> On 2022-02-08 01:13, Fujii Masao wrote:
>> AbortSubTransaction() should reset ActiveQueryDesc to
>> save_ActiveQueryDesc that ExecutorRun() set, instead of NULL?
>> Otherwise ActiveQueryDesc of top-level statement will be unavailable
>> after subtransaction is aborted in the nested statements.
>
> I once agreed above suggestion and made v20 patch making
> save_ActiveQueryDesc a global variable, but it caused segfault when
> calling pg_log_query_plan() after FreeQueryDesc().
>
> OTOH, doing some kind of reset of ActiveQueryDesc seems necessary
> since it also caused segfault when running pg_log_query_plan() during
> installcheck.
>
> There may be a better way, but resetting ActiveQueryDesc to NULL seems
> safe and simple.
> Of course it makes pg_log_query_plan() useless after a subtransaction
> is aborted.
> However, if it does not often happen that people want to know the
> running query's plan whose subtransaction is aborted, resetting
> ActiveQueryDesc to NULL would be acceptable.
>
> Attached is a patch that sets ActiveQueryDesc to NULL when a
> subtransaction is aborted.
>
> How do you think?
Attached new patch to fix patch apply failures.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v22-0001-log-running-query-plan.patch (25.0K, ../../[email protected]/2-v22-0001-log-running-query-plan.patch)
download | inline diff:
From 5be784278e8e7aeeeadf60a772afccda7b59e6e4 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 16 May 2022 12:02:16 +0900
Subject: [PATCH v22] Add function to log the plan of the query currently
running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat
---
doc/src/sgml/func.sgml | 49 +++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 140 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 ++++--
17 files changed, 315 insertions(+), 28 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 85ecc639fd..47f54b978b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28027,6 +28027,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -28141,6 +28160,36 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 47d80b0d25..13c9d727f7 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -60,6 +60,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2739,6 +2740,12 @@ AbortTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5060,6 +5067,12 @@ AbortSubTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 73da687d5d..59e010ea95 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -709,6 +709,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c461061fe9..339f8810cc 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +43,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1603,6 +1605,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by singal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1610,7 +1615,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -5006,3 +5011,134 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index ef2fd46092..a82ac87457 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -77,6 +77,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -301,10 +304,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 21a9fc0fdd..0b4a591ee7 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -657,6 +658,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 5f5803f681..9beb456f78 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8b6b5bbaaa..0afa690c53 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3387,6 +3388,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 1a5d29ac9b..ab2fdf80fd 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -37,6 +37,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index babe16f00a..45ba793681 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8093,6 +8093,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4550', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 666977fb1f..a59abbf53d 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -124,4 +125,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0af130fbc5..0ab8a14fee 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,6 +95,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index e4e1495b24..ae89a104ef 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index ee636900f3..dc8fd66466 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index f9a6882ecb..227d24b9d6 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,8 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
+extern PGDLLIMPORT QueryDesc *save_ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 01d1ad0b9a..c726c119b2 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 072fc36a1f..9f0bb8f813 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: 5bcc4d09332844ae369bcf99f18ace1c982b7301
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2022-09-08 10:19 ` torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2022-09-08 10:19 UTC (permalink / raw)
To: [email protected]
On 2022-05-16 17:02, torikoshia wrote:
> On 2022-03-09 19:04, torikoshia wrote:
>> On 2022-02-08 01:13, Fujii Masao wrote:
>>> AbortSubTransaction() should reset ActiveQueryDesc to
>>> save_ActiveQueryDesc that ExecutorRun() set, instead of NULL?
>>> Otherwise ActiveQueryDesc of top-level statement will be unavailable
>>> after subtransaction is aborted in the nested statements.
>>
>> I once agreed above suggestion and made v20 patch making
>> save_ActiveQueryDesc a global variable, but it caused segfault when
>> calling pg_log_query_plan() after FreeQueryDesc().
>>
>> OTOH, doing some kind of reset of ActiveQueryDesc seems necessary
>> since it also caused segfault when running pg_log_query_plan() during
>> installcheck.
>>
>> There may be a better way, but resetting ActiveQueryDesc to NULL seems
>> safe and simple.
>> Of course it makes pg_log_query_plan() useless after a subtransaction
>> is aborted.
>> However, if it does not often happen that people want to know the
>> running query's plan whose subtransaction is aborted, resetting
>> ActiveQueryDesc to NULL would be acceptable.
>>
>> Attached is a patch that sets ActiveQueryDesc to NULL when a
>> subtransaction is aborted.
>>
>> How do you think?
Attached new patch to fix patch apply failures again.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v23-0001-log-running-query-plan.patch (25.0K, ../../[email protected]/2-v23-0001-log-running-query-plan.patch)
download | inline diff:
From 5be784278e8e7aeeeadf60a772afccda7b59e6e4 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 5 Sep 2022 12:02:16 +0900
Subject: [PATCH v23] Add function to log the plan of the query currently
running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat
---
doc/src/sgml/func.sgml | 49 +++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 140 ++++++++++++++++++-
src/backend/executor/execMain.c | 10 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 ++++--
17 files changed, 315 insertions(+), 28 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 67eb380632..0e12f73b9c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25535,6 +25535,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25649,6 +25668,36 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 50f092d7eb..8a286f7559 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -60,6 +60,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2739,6 +2740,12 @@ AbortTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5072,6 +5079,12 @@ AbortSubTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 30a048f6b0..e347e6be90 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -713,6 +713,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 053d2ca5ae..9a361e87ac 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +43,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1625,6 +1627,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by singal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1632,7 +1637,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -5041,3 +5046,134 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index ef2fd46092..a82ac87457 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -77,6 +77,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -301,10 +304,17 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 21a9fc0fdd..0b4a591ee7 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -657,6 +658,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 5f5803f681..9beb456f78 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7bec4e4ff5..cb704d3e40 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -33,6 +33,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3377,6 +3378,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 1a5d29ac9b..ab2fdf80fd 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -37,6 +37,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a07e737a33..9a09d77f93 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8101,6 +8101,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4550', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 9ebde089ae..fc9f9f8e3f 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -125,4 +126,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 65cf4ba50f..fb57da0d19 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,6 +95,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index e4e1495b24..ae89a104ef 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index ee636900f3..dc8fd66466 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index f9a6882ecb..227d24b9d6 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,6 +21,8 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
+extern PGDLLIMPORT QueryDesc *save_ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 9f106c2a10..62196d0a39 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 639e9b352c..43b78a23fc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: 6bcda4a72123c3aa29fa3f03d952095675ad4468
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2022-09-19 08:47 ` Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: Алена Рыбакина @ 2022-09-19 08:47 UTC (permalink / raw)
To: torikoshia <[email protected]>; [email protected] <[email protected]>
<div><div>Hi,</div><div><div>I'm sorry,if this message is duplicated previous this one, but I'm not sure that the previous message is sent correctly. I sent it from email address <a href="mailto:[email protected]" rel="noopener noreferrer">[email protected]</a> and I couldn't send this one email from those address.</div><div>I like idea to create patch for logging query plan. After reviewing this code and notice some moments and I'd rather ask you some questions.</div></div><div><br />Firstly, I suggest some editing in the comment of commit. I think, it is turned out the more laconic and the same clear. I wrote it below since I can't think of any other way to add it.<br /><br />```<br />Currently, we have to wait for finishing of the query execution to check its plan.<br />This is not so convenient in investigation long-running queries on production<br />environments where we cannot use debuggers.<br /><br />To improve this situation there is proposed the patch containing the pg_log_query_plan()<br />function which request to log plan of the specified backend process.<br /><br />By default, only superusers are allowed to request log of the plan otherwise<br />allowing any users to issue this request could create cause lots of log messages<br />and it can lead to denial of service.<br /><br />At the next requesting CHECK_FOR_INTERRUPTS(), the target backend logs its plan at<br />LOG_SERVER_ONLY level and therefore this plan will appear in the server log only,<br />not to be sent to the client.<br />```<br /><br />Secondly, I have question about deleting USE_ASSERT_CHECKING in lock.h?<br />It supposed to have been checked in another placed of the code by matching values. I am worry about skipping errors due to untesting with assert option in the places where it (GetLockMethodLocalHash) participates and we won't able to get core file in segfault cases. I might not understand something, then can you please explain to me?<br /><br />Thirdly, I have incomprehension of the point why save_ActiveQueryDesc is declared in the pquery.h? I am seemed to save_ActiveQueryDesc to be used in an once time in the ExecutorRun function and its declaration superfluous. I added it in the attached patch.<br /><br />Fourthly, it seems to me there are not enough explanatory comments in the code. I also added them in the attached patch.<br /><br />Lastly, I have incomprehension about handling signals since have been unused it before. Could another signal disabled calling this signal to log query plan? I noticed this signal to be checked the latest in procsignal_sigusr1_handler function.</div></div><div> </div><div> </div><div>-- </div><div>Regards,</div><div>Alena Rybakina<br />Postgres Professional</div><div> </div><div> </div><div> </div><div>19.09.2022, 11:01, "torikoshia" <[email protected]>:</div><blockquote><p>On 2022-05-16 17:02, torikoshia wrote:</p><blockquote> On 2022-03-09 19:04, torikoshia wrote:<blockquote> On 2022-02-08 01:13, Fujii Masao wrote:<blockquote> AbortSubTransaction() should reset ActiveQueryDesc to<br /> save_ActiveQueryDesc that ExecutorRun() set, instead of NULL?<br /> Otherwise ActiveQueryDesc of top-level statement will be unavailable<br /> after subtransaction is aborted in the nested statements.</blockquote> <br /> I once agreed above suggestion and made v20 patch making<br /> save_ActiveQueryDesc a global variable, but it caused segfault when<br /> calling pg_log_query_plan() after FreeQueryDesc().<br /> <br /> OTOH, doing some kind of reset of ActiveQueryDesc seems necessary<br /> since it also caused segfault when running pg_log_query_plan() during<br /> installcheck.<br /> <br /> There may be a better way, but resetting ActiveQueryDesc to NULL seems<br /> safe and simple.<br /> Of course it makes pg_log_query_plan() useless after a subtransaction<br /> is aborted.<br /> However, if it does not often happen that people want to know the<br /> running query's plan whose subtransaction is aborted, resetting<br /> ActiveQueryDesc to NULL would be acceptable.<br /> <br /> Attached is a patch that sets ActiveQueryDesc to NULL when a<br /> subtransaction is aborted.<br /> <br /> How do you think?</blockquote></blockquote><p>Attached new patch to fix patch apply failures again.<br /> </p>--<br />Regards,<br /><br />--<br />Atsushi Torikoshi<br />NTT DATA CORPORATION</blockquote>
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
@ 2022-09-20 14:41 ` torikoshia <[email protected]>
2022-09-21 08:30 ` Re: RFC: Logging plan of the running query Alena Rybakina <[email protected]>
2022-12-06 18:41 ` Re: RFC: Logging plan of the running query Andres Freund <[email protected]>
0 siblings, 2 replies; 124+ messages in thread
From: torikoshia @ 2022-09-20 14:41 UTC (permalink / raw)
To: Алена Рыбакина <[email protected]>; +Cc: [email protected]
On 2022-09-19 17:47, Алена Рыбакина wrote:
Thanks for your review and comments!
> Hi,
>
> I'm sorry,if this message is duplicated previous this one, but I'm not
> sure that the previous message is sent correctly. I sent it from email
> address [email protected] and I couldn't send this one email
> from those address.
I've successfully received your mail from both [email protected]
and [email protected].
> I like idea to create patch for logging query plan. After reviewing
> this code and notice some moments and I'd rather ask you some
> questions.
>
> Firstly, I suggest some editing in the comment of commit. I think, it
> is
> turned out the more laconic and the same clear. I wrote it below since
> I
> can't think of any other way to add it.
> ```
> Currently, we have to wait for finishing of the query execution to
> check
> its plan.
> This is not so convenient in investigation long-running queries on
> production
> environments where we cannot use debuggers.
> To improve this situation there is proposed the patch containing the
> pg_log_query_plan()
> function which request to log plan of the specified backend process.
> By default, only superusers are allowed to request log of the plan
> otherwise
> allowing any users to issue this request could create cause lots of log
> messages
> and it can lead to denial of service.
>
> At the next requesting CHECK_FOR_INTERRUPTS(), the target backend logs
> its plan at
> LOG_SERVER_ONLY level and therefore this plan will appear in the server
> log only,
> not to be sent to the client.
Thanks, I have incorporated your comments.
Since the latter part of the original message comes from the commit
message of pg_log_backend_memory_contexts(43620e328617c), so I left it
as it was for consistency.
> Secondly, I have question about deleting USE_ASSERT_CHECKING in lock.h?
> It supposed to have been checked in another placed of the code by
> matching values. I am worry about skipping errors due to untesting with
> assert option in the places where it (GetLockMethodLocalHash)
> participates and we won't able to get core file in segfault cases. I
> might not understand something, then can you please explain to me?
Since GetLockMethodLocalHash() is only used for assertions, this is only
defined when USE_ASSERT_CHECKING is enabled.
This patch uses GetLockMethodLocalHash() not only for the assertion
purpose, so I removed "ifdef USE_ASSERT_CHECKING" for this function.
I belive it does not lead to skip errors.
> Thirdly, I have incomprehension of the point why save_ActiveQueryDesc
> is
> declared in the pquery.h? I am seemed to save_ActiveQueryDesc to be
> used
> in an once time in the ExecutorRun function and its declaration
> superfluous. I added it in the attached patch.
Exactly.
> Fourthly, it seems to me there are not enough explanatory comments in
> the code. I also added them in the attached patch.
Thanks!
| + /*
| + * Save value of ActiveQueryDesc before having called
| + * ExecutorRun_hook function due to having reset by
| + * AbortSubTransaction.
| + */
| +
| save_ActiveQueryDesc = ActiveQueryDesc;
| ActiveQueryDesc = queryDesc;
|
| @@ -314,6 +320,7 @@ ExecutorRun(QueryDesc *queryDesc,
| else
| standard_ExecutorRun(queryDesc, direction, count,
execute_once);
|
| + /* We set the actual value of ActiveQueryDesc */
| ActiveQueryDesc = save_ActiveQueryDesc;
Since these processes are needed for nested queries, not only for
AbortSubTransaction[1], added comments on it.
| +/* Function to handle the signal to output the query plan. */
| extern void HandleLogQueryPlanInterrupt(void);
I feel this comment is unnecessary since the explanation of
HandleLogQueryPlanInterrupt() is written in explain.c and no functions
in explain.h have comments in it.
> Lastly, I have incomprehension about handling signals since have been
> unused it before. Could another signal disabled calling this signal to
> log query plan? I noticed this signal to be checked the latest in
> procsignal_sigusr1_handler function.
Are you concerned that one signal will not be processed when multiple
signals are sent in succession?
AFAIU both of them are processed since SendProcSignal flags
ps_signalFlags for each signal.
```
SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
{
volatile ProcSignalSlot *slot;
...(snip)...
278 if (slot->pss_pid == pid)
279 {
280 /* Atomically set the proper flag */
281 slot->pss_signalFlags[reason] = true;
282 /* Send signal */
283 return kill(pid, SIGUSR1);
```
Comments of ProcSignalReason also say 'We can cope with concurrent
signals for different reasons'.
```C
/*
* Reasons for signaling a Postgres child process (a backend or an
auxiliary
* process, like checkpointer). We can cope with concurrent signals
for different
* reasons. However, if the same reason is signaled multiple times in
quick
* succession, the process is likely to observe only one notification
of it.
* This is okay for the present uses.
...
typedef enum
{
PROCSIG_CATCHUP_INTERRUPT, /* sinval catchup interrupt */
PROCSIG_NOTIFY_INTERRUPT, /* listen/notify interrupt */
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel
backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for
shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory
contexts */
PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
query */
...
} ProcSignalReason;
```
[1]
https://www.postgresql.org/message-id/8b53b32f-26cc-0531-4ac0-27310e0bef4b%40oss.nttdata.com
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v24-0001-log-running-query-plan.patch (25.1K, ../../[email protected]/2-v24-0001-log-running-query-plan.patch)
download | inline diff:
From a0d2179826a0fa224eaf37ca00d14954b76fde6b Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Tue, 20 Sep 2022 21:52:41 +0900
Subject: [PATCH v24] Add function to log the plan of the query
currently running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat, Alena Rybakina
---
doc/src/sgml/func.sgml | 49 +++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 140 ++++++++++++++++++-
src/backend/executor/execMain.c | 14 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 ++++--
17 files changed, 318 insertions(+), 29 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e1fe4fec57..bd80d062c1 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25535,6 +25535,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25649,6 +25668,36 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 2bb975943c..fb698ac007 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -60,6 +60,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2739,6 +2740,12 @@ AbortTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5072,6 +5079,12 @@ AbortSubTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 30a048f6b0..e347e6be90 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -713,6 +713,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 053d2ca5ae..9a361e87ac 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -40,7 +43,6 @@
#include "utils/typcache.h"
#include "utils/xml.h"
-
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -1625,6 +1627,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by signal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1632,7 +1637,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -5041,3 +5046,134 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index d78862e660..1ce107a069 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -77,6 +77,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckRTPerms() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -301,10 +304,21 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ /*
+ * Update ActiveQueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 21a9fc0fdd..0b4a591ee7 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -657,6 +658,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 5f5803f681..9beb456f78 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 35eff28bd3..053d460ab2 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -33,6 +33,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3377,6 +3378,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 1a5d29ac9b..ab2fdf80fd 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -37,6 +37,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a07e737a33..9a09d77f93 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8101,6 +8101,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '4550', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 9ebde089ae..fc9f9f8e3f 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -125,4 +126,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index ee48e392ed..960366bf93 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,6 +95,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index e4e1495b24..ae89a104ef 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index ee636900f3..dc8fd66466 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index f9a6882ecb..4a0e94ab1a 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,7 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 9f106c2a10..62196d0a39 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 639e9b352c..43b78a23fc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2022-09-21 08:30 ` Alena Rybakina <[email protected]>
2022-09-22 14:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Alena Rybakina @ 2022-09-21 08:30 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]
Ok, I get it.
> Since GetLockMethodLocalHash() is only used for assertions, this is
> only defined when USE_ASSERT_CHECKING is enabled. This patch uses
> GetLockMethodLocalHash() not only for the assertion purpose, so I
> removed "ifdef USE_ASSERT_CHECKING" for this function. I belive it
> does not lead to skip errors.
Agree.
> Since these processes are needed for nested queries, not only for
> AbortSubTransaction[1], added comments on it.
I also noticed it. However I also discovered that above function
declarations to be aplied for explain command and used to be printed
details of the executed query.
We have a similar task to print the plan of an interrupted process
making a request for a specific pid.
In short, I think, this task is different and for separating these parts
I added this comment.
> I feel this comment is unnecessary since the explanation of
> HandleLogQueryPlanInterrupt() is written in explain.c and no functions
> in explain.h have comments in it.
Yes, I was worried about it. I understood it, thank for explaining.
>
> AFAIU both of them are processed since SendProcSignal flags
> ps_signalFlags for each signal.
>
> ```
> SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
> {
> volatile ProcSignalSlot *slot;
> ...(snip)...
> 278 if (slot->pss_pid == pid)
> 279 {
> 280 /* Atomically set the proper flag */
> 281 slot->pss_signalFlags[reason] = true;
> 282 /* Send signal */
> 283 return kill(pid, SIGUSR1);
> ```
>
> Comments of ProcSignalReason also say 'We can cope with concurrent
> signals for different reasons'.
>
> ```C
> /*
> * Reasons for signaling a Postgres child process (a backend or an
> auxiliary
> * process, like checkpointer). We can cope with concurrent signals for
> different
> * reasons. However, if the same reason is signaled multiple times in
> quick
> * succession, the process is likely to observe only one notification
> of it.
> * This is okay for the present uses.
> ...
> typedef enum
> {
> PROCSIG_CATCHUP_INTERRUPT, /* sinval catchup interrupt */
> PROCSIG_NOTIFY_INTERRUPT, /* listen/notify interrupt */
> PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
> PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for
> shutdown */
> PROCSIG_BARRIER, /* global barrier interrupt */
> PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
> PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current
> query */
> ...
> } ProcSignalReason;
> ```
>
>
> [1]
> https://www.postgresql.org/message-id/8b53b32f-26cc-0531-4ac0-27310e0bef4b%40oss.nttdata.com
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-21 08:30 ` Re: RFC: Logging plan of the running query Alena Rybakina <[email protected]>
@ 2022-09-22 14:12 ` torikoshia <[email protected]>
2022-09-23 09:43 ` Re: RFC: Logging plan of the running query Alena Rybakina <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2022-09-22 14:12 UTC (permalink / raw)
To: Alena Rybakina <[email protected]>; +Cc: [email protected]
On 2022-09-21 17:30, Alena Rybakina wrote:
Thanks for your reply!
> I also noticed it. However I also discovered that above function
> declarations to be aplied for explain command and used to be printed
> details of the executed query.
>
> We have a similar task to print the plan of an interrupted process
> making a request for a specific pid.
>
> In short, I think, this task is different and for separating these
> parts I added this comment.
I'm not sure I understand your comment correctly, do you mean
HandleLogQueryPlanInterrupt() should not be placed in explain.c?
It may be so.
However, given that ProcesLogMemoryContextInterrupt(), which similarly
handles interrupts for pg_log_backend_memory_contexts(), is located in
mcxt.c, I also think current location might be acceptable.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-21 08:30 ` Re: RFC: Logging plan of the running query Alena Rybakina <[email protected]>
2022-09-22 14:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2022-09-23 09:43 ` Alena Rybakina <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: Alena Rybakina @ 2022-09-23 09:43 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: [email protected]
Sorry, I wrote confusingly at that time.
No, I suggested adding comment about the explanation of
HandleLogQueryPlanInterrupt() only in the explain.h and not removing
from the explain.c.
I seemed to be necessary separating declaration function for 'explaining
feature' of executed query from our logging plan of the running query
interrupts function. But now, I doubt it.
> I'm not sure I understand your comment correctly, do you mean
> HandleLogQueryPlanInterrupt() should not be placed in explain.c?
Thank you for having reminded about this function and I looked at
ProcessLogMemoryContextInterrupt() declaration. I'm noticed comments in
the memutils.h are missed tooю
Description of this function is written only in mcxt.c.
> However, given that ProcesLogMemoryContextInterrupt(), which similarly
> handles interrupts for pg_log_backend_memory_contexts(), is located in
> mcxt.c, I also think current location might be acceptable.
So I think you are right and the comment about the explanation of
HandleLogQueryPlanInterrupt() written in explain.h is redundant.
> I feel this comment is unnecessary since the explanation of
> HandleLogQueryPlanInterrupt() is written in explain.c and no functions
> in explain.h have comments in it.
Regards,
--
Alena Rybakina
Postgres Professional
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2022-12-06 18:41 ` Andres Freund <[email protected]>
2022-12-08 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
1 sibling, 1 reply; 124+ messages in thread
From: Andres Freund @ 2022-12-06 18:41 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Алена Рыбакина <[email protected]>; [email protected]
Hi,
This patch does not currently build, due to a conflicting oid:
https://cirrus-ci.com/task/4638460594618368?logs=build#L108
[17:26:44.602] /usr/bin/perl ../src/include/catalog/../../backend/catalog/genbki.pl --include-path=../src/include --set-version=16 --output=src/include/catalog ../src/include/catalog/pg_proc.h ../src/include/catalog/pg_type.h ../src/include/catalog/pg_attribute.h ../src/include/catalog/pg_class.h ../src/include/catalog/pg_attrdef.h ../src/include/catalog/pg_constraint.h ../src/include/catalog/pg_inherits.h ../src/include/catalog/pg_index.h ../src/include/catalog/pg_operator.h ../src/include/catalog/pg_opfamily.h ../src/include/catalog/pg_opclass.h ../src/include/catalog/pg_am.h ../src/include/catalog/pg_amop.h ../src/include/catalog/pg_amproc.h ../src/include/catalog/pg_language.h ../src/include/catalog/pg_largeobject_metadata.h ../src/include/catalog/pg_largeobject.h ../src/include/catalog/pg_aggregate.h ../src/include/catalog/pg_statistic.h ../src/include/catalog/pg_statistic_ext.h ../src/include/catalog/pg_statistic_ext_data.h ../src/include/catalog/pg_rewrite.h ../src/include/catalog/pg_trigger.h ../src/include/catalog/pg_event_trigger.h ../src/include/catalog/pg_description.h ../src/include/catalog/pg_cast.h ../src/include/catalog/pg_enum.h ../src/include/catalog/pg_namespace.h ../src/include/catalog/pg_conversion.h ../src/include/catalog/pg_depend.h ../src/include/catalog/pg_database.h ../src/include/catalog/pg_db_role_setting.h ../src/include/catalog/pg_tablespace.h ../src/include/catalog/pg_authid.h ../src/include/catalog/pg_auth_members.h ../src/include/catalog/pg_shdepend.h ../src/include/catalog/pg_shdescription.h ../src/include/catalog/pg_ts_config.h ../src/include/catalog/pg_ts_config_map.h ../src/include/catalog/pg_ts_dict.h ../src/include/catalog/pg_ts_parser.h ../src/include/catalog/pg_ts_template.h ../src/include/catalog/pg_extension.h ../src/include/catalog/pg_foreign_data_wrapper.h ../src/include/catalog/pg_foreign_server.h ../src/include/catalog/pg_user_mapping.h ../src/include/catalog/pg_foreign_table.h ../src/include/catalog/pg_policy.h ../src/include/catalog/pg_replication_origin.h ../src/include/catalog/pg_default_acl.h ../src/include/catalog/pg_init_privs.h ../src/include/catalog/pg_seclabel.h ../src/include/catalog/pg_shseclabel.h ../src/include/catalog/pg_collation.h ../src/include/catalog/pg_parameter_acl.h ../src/include/catalog/pg_partitioned_table.h ../src/include/catalog/pg_range.h ../src/include/catalog/pg_transform.h ../src/include/catalog/pg_sequence.h ../src/include/catalog/pg_publication.h ../src/include/catalog/pg_publication_namespace.h ../src/include/catalog/pg_publication_rel.h ../src/include/catalog/pg_subscription.h ../src/include/catalog/pg_subscription_rel.h
[17:26:44.602] Duplicate OIDs detected:
[17:26:44.602] 4550
[17:26:44.602] found 1 duplicate OID(s) in catalog data
I suggest you choose a random oid out of the "development purposes" range:
* OIDs 1-9999 are reserved for manual assignment (see .dat files in
* src/include/catalog/). Of these, 8000-9999 are reserved for
* development purposes (such as in-progress patches and forks);
* they should not appear in released versions.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-12-06 18:41 ` Re: RFC: Logging plan of the running query Andres Freund <[email protected]>
@ 2022-12-08 05:10 ` torikoshia <[email protected]>
2023-06-02 17:51 ` Re: RFC: Logging plan of the running query James Coleman <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2022-12-08 05:10 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: [email protected]
On 2022-12-07 03:41, Andres Freund wrote:
> Hi,
>
> This patch does not currently build, due to a conflicting oid:
>
> I suggest you choose a random oid out of the "development purposes"
> range:
Thanks for your advice!
Attached updated patch.
BTW, since this patch depends on ProcessInterrupts() and EXPLAIN codes
which is used in auto_explain, I'm feeling that the following discussion
also applies to this patch.
> --
> https://www.postgresql.org/message-id/CA%2BTgmoYW_rSOW4JMQ9_0Df9PKQ%3DsQDOKUGA4Gc9D8w4wui8fSA%40mail...
>
> explaining a query is a pretty
> complicated operation that involves catalog lookups and lots of
> complicated stuff, and I don't think that it would be safe to do all
> of that at any arbitrary point in the code where ProcessInterrupts()
> happened to fire.
If I can't come up with some workaround during the next Commitfest, I'm
going to withdraw this proposal.
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
Attachments:
[text/x-diff] v25-0001-log-running-query-plan.patch (24.9K, ../../[email protected]/2-v25-0001-log-running-query-plan.patch)
download | inline diff:
From a0d2179826a0fa224eaf37ca00d14954b76fde6b Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Thu, 8 Dec 2022 12:42:22 +0900
Subject: [PATCH v25] Add function to log the plan of the query
currently running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat, Alena Rybakina
---
doc/src/sgml/func.sgml | 49 +++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 139 ++++++++++++++++++-
src/backend/executor/execMain.c | 14 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 ++++--
17 files changed, 318 insertions(+), 28 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e57ffce971..d39a4e50a0 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25536,6 +25536,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25756,6 +25775,36 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 8086b857b9..08cb48ece1 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -60,6 +60,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2739,6 +2740,12 @@ AbortTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5090,6 +5097,12 @@ AbortSubTransaction(void)
*/
PG_SETMASK(&UnBlockSig);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 52517a6531..5ada291ce6 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -739,6 +739,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..ff46433f05 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -1625,6 +1628,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by signal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1632,7 +1638,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -5041,3 +5047,134 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 872b879387..01d3f22c07 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -78,6 +78,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckPermissions() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -302,10 +305,21 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ /*
+ * Update ActiveQueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 7767657f27..aabb2c3fb6 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/walsender.h"
@@ -657,6 +658,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 3d1049cf75..35b8891063 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 3082093d1e..e99b02ca81 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -33,6 +33,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3379,6 +3380,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 00bceec8fa..87d50de9f8 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -37,6 +37,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f9301b2627..14cc65bb6b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8135,6 +8135,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 9ebde089ae..fc9f9f8e3f 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -59,6 +59,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -125,4 +126,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 795182fa51..814d15d1d8 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,6 +95,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index e4e1495b24..ae89a104ef 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -561,9 +561,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index ee636900f3..dc8fd66466 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index f9a6882ecb..4a0e94ab1a 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,7 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 88bb696ded..e43c0f7252 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index b07e9e8dbb..3aa6cea941 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: bf07ab492c461460b4a69279abb2ef996b4f67ec
--
2.27.0
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-12-06 18:41 ` Re: RFC: Logging plan of the running query Andres Freund <[email protected]>
2022-12-08 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2023-06-02 17:51 ` James Coleman <[email protected]>
2023-06-05 08:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: James Coleman @ 2023-06-02 17:51 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Andres Freund <[email protected]>; [email protected]; Greg Stark <[email protected]>; Ronan Dunklau <[email protected]>; [email protected]; Heikki Linnakangas <[email protected]>
Hello,
Thanks for working on this patch!
On Thu, Dec 8, 2022 at 12:10 AM torikoshia <[email protected]> wrote:
>
> BTW, since this patch depends on ProcessInterrupts() and EXPLAIN codes
> which is used in auto_explain, I'm feeling that the following discussion
> also applies to this patch.
>
> > --
> > https://www.postgresql.org/message-id/CA%2BTgmoYW_rSOW4JMQ9_0Df9PKQ%3DsQDOKUGA4Gc9D8w4wui8fSA%40mail...
> >
> > explaining a query is a pretty
> > complicated operation that involves catalog lookups and lots of
> > complicated stuff, and I don't think that it would be safe to do all
> > of that at any arbitrary point in the code where ProcessInterrupts()
> > happened to fire.
>
> If I can't come up with some workaround during the next Commitfest, I'm
> going to withdraw this proposal.
While at PGCon this week I'd brought up this idea with a few people
without realizing you'd already worked on it previously, and then
after I discovered this thread several of us (Greg, Ronan, David,
Heikki, and myself -- all cc'd) discussed the safety concerns over
dinner last night.
Our conclusion was that all of the concerns we could come up with (for
example, walking though the code for ExplainTargetRel() and discussing
the relevant catalog and syscache accesses) all applied specifically
to Robert's concerns about running explain inside an aborted
transaction. After all, I'd originally started that thread to ask
about running auto-explain after a query timeout.
To put it positively: we believe that, for example, catalog accesses
inside CHECK_FOR_INTERRUPTS() -- assuming that the CFI call is inside
an existing valid transaction/query state, as it would be for this
patch -- are safe. If there were problems, then those problems are
likely bugs we already have in other CFI cases.
Another concern Robert raised may apply here: what if a person tries
to cancel a query when we're explaining? I believe that's a reasonable
question to ask, but I believe it's a trade-off that's worth making
for the (significant) introspection benefits this patch would provide.
On to the patch itself: overall I think it looks like it's in pretty
good shape. I also noticed we don't have any tests (I assume it'd have
to be TAP tests) of the actual output happening, and I think it would
be worth adding that.
Are you interested in re-opening this patch? I'd be happy to provide
further review and help to try to push this along.
I've rebased the patch and attached as v26.
Thanks,
James Coleman
Attachments:
[application/octet-stream] v26-0001-Add-function-to-log-the-plan-of-the-query.patch (25.0K, ../../CAAaqYe9euUZD8bkjXTVcD9e4n5c7kzHzcvuCJXt-xds9X4c7Fw@mail.gmail.com/2-v26-0001-Add-function-to-log-the-plan-of-the-query.patch)
download | inline diff:
From d4aa91ce915e2f0ebe44567bdaa7d2364db7534d Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Thu, 8 Dec 2022 12:42:22 +0900
Subject: [PATCH v26] Add function to log the plan of the query
currently running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat, Alena Rybakina
---
doc/src/sgml/func.sgml | 49 +++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 139 ++++++++++++++++++-
src/backend/executor/execMain.c | 14 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 ++++--
17 files changed, 318 insertions(+), 28 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5a47ce4343..ab425900f9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26191,6 +26191,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -26411,6 +26430,36 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 8daaa535ed..e059e46331 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -61,6 +61,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2750,6 +2751,12 @@ AbortTransaction(void)
*/
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5108,6 +5115,12 @@ AbortSubTransaction(void)
*/
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 07c0d89c4f..80792913bd 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -722,6 +722,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 15f9bddcdf..3f74e8bad5 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -1639,6 +1642,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by signal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1646,7 +1652,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -5052,3 +5058,134 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index c76fdf59ec..bf21b83576 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -78,6 +78,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckPermissions() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -303,10 +306,21 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ /*
+ * Update ActiveQueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index c85cb5cc18..8911977340 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/logicalworker.h"
@@ -658,6 +659,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 193f50fc0f..cba49b6880 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 01b6cc1f7d..25a2775364 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -36,6 +36,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3445,6 +3446,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
HandleParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 011ec18015..c451f84e07 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -37,6 +37,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6996073989..6511c36627 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8223,6 +8223,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..3715dd6cd1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -60,6 +60,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -126,4 +127,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 14bd574fc2..f805daec16 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,6 +95,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 8575bea25c..63f0454378 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -569,9 +569,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 2f52100b00..0407a1433b 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..a6ae947a0f 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,7 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c669948370..0b98f8a972 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index b57f01f3e9..84327408d8 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
2.39.2 (Apple Git-143)
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-12-06 18:41 ` Re: RFC: Logging plan of the running query Andres Freund <[email protected]>
2022-12-08 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2023-06-02 17:51 ` Re: RFC: Logging plan of the running query James Coleman <[email protected]>
@ 2023-06-05 08:30 ` torikoshia <[email protected]>
2023-06-05 18:26 ` Re: RFC: Logging plan of the running query James Coleman <[email protected]>
0 siblings, 1 reply; 124+ messages in thread
From: torikoshia @ 2023-06-05 08:30 UTC (permalink / raw)
To: James Coleman <[email protected]>; +Cc: Andres Freund <[email protected]>; [email protected]; Greg Stark <[email protected]>; Ronan Dunklau <[email protected]>; [email protected]; Heikki Linnakangas <[email protected]>
On 2023-06-03 02:51, James Coleman wrote:
> Hello,
>
> Thanks for working on this patch!
>
> On Thu, Dec 8, 2022 at 12:10 AM torikoshia <[email protected]>
> wrote:
>>
>> BTW, since this patch depends on ProcessInterrupts() and EXPLAIN codes
>> which is used in auto_explain, I'm feeling that the following
>> discussion
>> also applies to this patch.
>>
>> > --
>> > https://www.postgresql.org/message-id/CA%2BTgmoYW_rSOW4JMQ9_0Df9PKQ%3DsQDOKUGA4Gc9D8w4wui8fSA%40mail...
>> >
>> > explaining a query is a pretty
>> > complicated operation that involves catalog lookups and lots of
>> > complicated stuff, and I don't think that it would be safe to do all
>> > of that at any arbitrary point in the code where ProcessInterrupts()
>> > happened to fire.
>>
>> If I can't come up with some workaround during the next Commitfest,
>> I'm
>> going to withdraw this proposal.
>
> While at PGCon this week I'd brought up this idea with a few people
> without realizing you'd already worked on it previously, and then
> after I discovered this thread several of us (Greg, Ronan, David,
> Heikki, and myself -- all cc'd) discussed the safety concerns over
> dinner last night.
>
> Our conclusion was that all of the concerns we could come up with (for
> example, walking though the code for ExplainTargetRel() and discussing
> the relevant catalog and syscache accesses) all applied specifically
> to Robert's concerns about running explain inside an aborted
> transaction. After all, I'd originally started that thread to ask
> about running auto-explain after a query timeout.
>
> To put it positively: we believe that, for example, catalog accesses
> inside CHECK_FOR_INTERRUPTS() -- assuming that the CFI call is inside
> an existing valid transaction/query state, as it would be for this
> patch -- are safe. If there were problems, then those problems are
> likely bugs we already have in other CFI cases.
Thanks a lot for the discussion and sharing it!
I really appreciate it.
BTW I'm not sure whether all the CFI are called in valid transaction,
do you think we should check each of them?
> Another concern Robert raised may apply here: what if a person tries
> to cancel a query when we're explaining? I believe that's a reasonable
> question to ask, but I believe it's a trade-off that's worth making
> for the (significant) introspection benefits this patch would provide.
Is the concern here limited to the case where explain code goes crazy
as Robert pointed out?
If so, this may be a trade-off worth doing.
I am a little concerned about whether there will be cases where the
explain code is not problematic but just takes long time.
> On to the patch itself: overall I think it looks like it's in pretty
> good shape. I also noticed we don't have any tests (I assume it'd have
> to be TAP tests) of the actual output happening, and I think it would
> be worth adding that.
Checking the log output may be better, but I didn't add it since there
is only a little content that can be checked, and similar function
pg_log_backend_memory_contexts() does not do such type of tests.
> Are you interested in re-opening this patch? I'd be happy to provide
> further review and help to try to push this along.
Sure, I'm going to re-open this.
> I've rebased the patch and attached as v26.
Thanks again for your work!
--
Regards,
--
Atsushi Torikoshi
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-12-06 18:41 ` Re: RFC: Logging plan of the running query Andres Freund <[email protected]>
2022-12-08 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2023-06-02 17:51 ` Re: RFC: Logging plan of the running query James Coleman <[email protected]>
2023-06-05 08:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
@ 2023-06-05 18:26 ` James Coleman <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: James Coleman @ 2023-06-05 18:26 UTC (permalink / raw)
To: torikoshia <[email protected]>; +Cc: Andres Freund <[email protected]>; [email protected]; Greg Stark <[email protected]>; Ronan Dunklau <[email protected]>; [email protected]; Heikki Linnakangas <[email protected]>
On Mon, Jun 5, 2023 at 4:30 AM torikoshia <[email protected]> wrote:
>
> On 2023-06-03 02:51, James Coleman wrote:
> > Hello,
> >
> > Thanks for working on this patch!
Sure thing! I'm *very interested* in seeing this available, and I
think it paves the way for some additional features later on...
> > On Thu, Dec 8, 2022 at 12:10 AM torikoshia <[email protected]>
> ...
> > To put it positively: we believe that, for example, catalog accesses
> > inside CHECK_FOR_INTERRUPTS() -- assuming that the CFI call is inside
> > an existing valid transaction/query state, as it would be for this
> > patch -- are safe. If there were problems, then those problems are
> > likely bugs we already have in other CFI cases.
>
> Thanks a lot for the discussion and sharing it!
> I really appreciate it.
>
> BTW I'm not sure whether all the CFI are called in valid transaction,
> do you think we should check each of them?
I kicked off the regressions tests with a call to
ProcessLogQueryPlanInterrupt() in every single CHECK_FOR_INTERRUPTS()
call. Several hours and 52 GB of logs later I have confirmed that
(with the attached revision) at the very least the regression test
suite can't trigger any kind of failures regardless of when we trigger
this. The existing code in the patch for only running the explain when
there's an active query handling that.
> > Another concern Robert raised may apply here: what if a person tries
> > to cancel a query when we're explaining? I believe that's a reasonable
> > question to ask, but I believe it's a trade-off that's worth making
> > for the (significant) introspection benefits this patch would provide.
>
> Is the concern here limited to the case where explain code goes crazy
> as Robert pointed out?
> If so, this may be a trade-off worth doing.
> I am a little concerned about whether there will be cases where the
> explain code is not problematic but just takes long time.
The explain code could take a long time in some rare cases (e.g., we
discovered a bug a few years back with the planning code that actually
descends an index to find the max value), but I think the trade-off is
worth it.
> > On to the patch itself: overall I think it looks like it's in pretty
> > good shape. I also noticed we don't have any tests (I assume it'd have
> > to be TAP tests) of the actual output happening, and I think it would
> > be worth adding that.
>
> Checking the log output may be better, but I didn't add it since there
> is only a little content that can be checked, and similar function
> pg_log_backend_memory_contexts() does not do such type of tests.
Fair enough. I still think that would be an improvement here, but
others could also weigh in.
> > Are you interested in re-opening this patch? I'd be happy to provide
> > further review and help to try to push this along.
> Sure, I'm going to re-open this.
I've attached v27. The important change here in 0001 is that it
guarantees the interrupt handler is re-entrant, since that was a bug
exposed by my testing. I've also included 0002 which is only meant for
testing (it attempts to log in the plan in every
CHECK_FOR_INTERRUPTS() call).
Regards,
James
Attachments:
[application/octet-stream] v27-0001-Add-function-to-log-the-plan-of-the-query.patch (25.4K, ../../CAAaqYe8LXVXQhYy3yT0QOHUymdM=uha0dJ0=BEPzVAx2nG1gsw@mail.gmail.com/2-v27-0001-Add-function-to-log-the-plan-of-the-query.patch)
download | inline diff:
From fce1bc8e275682441691ce2bacaca3d413259abd Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Thu, 8 Dec 2022 12:42:22 +0900
Subject: [PATCH v27 1/2] Add function to log the plan of the query
currently running on the backend.
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat, Alena Rybakina
Co-authored-by: James Coleman <[email protected]>
---
doc/src/sgml/func.sgml | 49 ++++++
src/backend/access/transam/xact.c | 13 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 152 ++++++++++++++++++-
src/backend/executor/execMain.c | 14 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 3 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/test/regress/expected/misc_functions.out | 54 +++++--
src/test/regress/sql/misc_functions.sql | 41 +++--
17 files changed, 331 insertions(+), 28 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5a47ce4343..ab425900f9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26191,6 +26191,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -26411,6 +26430,36 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 8daaa535ed..e059e46331 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -61,6 +61,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2750,6 +2751,12 @@ AbortTransaction(void)
*/
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
@@ -5108,6 +5115,12 @@ AbortSubTransaction(void)
*/
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
+ /*
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 07c0d89c4f..80792913bd 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -722,6 +722,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 15f9bddcdf..f29e7ea517 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,6 +29,8 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
@@ -1639,6 +1642,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by signal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1646,7 +1652,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -5052,3 +5058,147 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+bool ProcessLogQueryPlanInterruptActive = false;
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ LogQueryPlanPending = false;
+
+ /* Cannot re-enter. */
+ if (ProcessLogQueryPlanInterruptActive)
+ return;
+
+ ProcessLogQueryPlanInterruptActive = true;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock confilcts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+ }
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, ActiveQueryDesc);
+ ExplainPrintPlan(es, ActiveQueryDesc);
+ ExplainPrintJITSummary(es, ActiveQueryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index c76fdf59ec..bf21b83576 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -78,6 +78,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckPermissions() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -303,10 +306,21 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ /*
+ * Update ActiveQueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index c85cb5cc18..8911977340 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/logicalworker.h"
@@ -658,6 +659,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 193f50fc0f..cba49b6880 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -614,17 +614,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 01b6cc1f7d..25a2775364 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -36,6 +36,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3445,6 +3446,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
HandleParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 011ec18015..c451f84e07 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -37,6 +37,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6996073989..6511c36627 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8223,6 +8223,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..3715dd6cd1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -60,6 +60,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -126,4 +127,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 14bd574fc2..f805daec16 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,6 +95,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 8575bea25c..63f0454378 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -569,9 +569,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 2f52100b00..0407a1433b 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..a6ae947a0f 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,7 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index c669948370..0b98f8a972 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index b57f01f3e9..84327408d8 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
2.20.1
[application/octet-stream] v27-0002-Testing-attempt-logging-plan-on-ever-CFI-call.patch (891B, ../../CAAaqYe8LXVXQhYy3yT0QOHUymdM=uha0dJ0=BEPzVAx2nG1gsw@mail.gmail.com/3-v27-0002-Testing-attempt-logging-plan-on-ever-CFI-call.patch)
download | inline diff:
From 319d76b296f9ff5ae08569417cdf7f3ad2d1da7d Mon Sep 17 00:00:00 2001
From: jcoleman <[email protected]>
Date: Mon, 5 Jun 2023 18:08:49 +0000
Subject: [PATCH v27 2/2] Testing: attempt logging plan on ever CFI call
Co-authored-by: James Coleman <[email protected]>
---
src/include/miscadmin.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index f805daec16..c6c6ffd204 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -118,9 +118,11 @@ extern void ProcessInterrupts(void);
unlikely(InterruptPending))
#endif
+extern void ProcessLogQueryPlanInterrupt(void);
/* Service interrupt, if one is pending and it's safe to service it now */
#define CHECK_FOR_INTERRUPTS() \
do { \
+ ProcessLogQueryPlanInterrupt(); \
if (INTERRUPTS_PENDING_CONDITION()) \
ProcessInterrupts(); \
} while(0)
--
2.20.1
^ permalink raw reply [nested|flat] 124+ messages in thread
* RFC: Logging plan of the running query
@ 2022-09-16 18:51 a.rybakina <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: a.rybakina @ 2022-09-16 18:51 UTC (permalink / raw)
To: [email protected]; +Cc: pgsql-hackers
Hi,
I liked this idea and after reviewing code I noticed some moments and
I'd rather ask you some questions.
Firstly, I suggest some editing in the comment of commit. I think, it is
turned out the more laconic and the same clear. I wrote it below since I
can't think of any other way to add it.
```
Currently, we have to wait for finishing of the query execution to check
its plan.
This is not so convenient in investigation long-running queries on
production
environments where we cannot use debuggers.
To improve this situation there is proposed the patch containing the
pg_log_query_plan()
function which request to log plan of the specified backend process.
By default, only superusers are allowed to request log of the plan
otherwise
allowing any users to issue this request could create cause lots of log
messages
and it can lead to denial of service.
At the next requesting CHECK_FOR_INTERRUPTS(), the target backend logs
its plan at
LOG_SERVER_ONLY level and therefore this plan will appear in the server
log only,
not to be sent to the client.
```
Secondly, I have question about deleting USE_ASSERT_CHECKING in lock.h?
It supposed to have been checked in another placed of the code by
matching values. I am worry about skipping errors due to untesting with
assert option in the places where it (GetLockMethodLocalHash)
participates and we won't able to get core file in segfault cases. I
might not understand something, then can you please explain to me?
Thirdly, I have incomprehension of the point why save_ActiveQueryDesc is
declared in the pquery.h? I am seemed to save_ActiveQueryDesc to be used
in an once time in the ExecutorRun function and its declaration
superfluous. I added it in the attached patch.
Fourthly, it seems to me there are not enough explanatory comments in
the code. I also added them in the attached patch.
Lastly, I have incomprehension about handling signals since have been
unused it before. Could another signal disabled calling this signal to
log query plan? I noticed this signal to be checked the latest in
procsignal_sigusr1_handler function.
Regards,
--
Alena Rybakina
Postgres Professional
Attachments:
[text/x-patch] diff_query_plan.patch (1.7K, ../../[email protected]/2-diff_query_plan.patch)
download | inline diff:
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index a82ac87457e..65b692b0ddf 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -306,6 +306,12 @@ ExecutorRun(QueryDesc *queryDesc,
{
QueryDesc *save_ActiveQueryDesc;
+ /*
+ * Save value of ActiveQueryDesc before having called
+ * ExecutorRun_hook function due to having reset by
+ * AbortSubTransaction.
+ */
+
save_ActiveQueryDesc = ActiveQueryDesc;
ActiveQueryDesc = queryDesc;
@@ -314,6 +320,7 @@ ExecutorRun(QueryDesc *queryDesc,
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+ /* We set the actual value of ActiveQueryDesc */
ActiveQueryDesc = save_ActiveQueryDesc;
}
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index fc9f9f8e3f0..8e7ce3c976f 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -126,6 +126,7 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+/* Function to handle the signal to output the query plan. */
extern void HandleLogQueryPlanInterrupt(void);
extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 227d24b9d60..d04de1f5566 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -22,7 +22,6 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
-extern PGDLLIMPORT QueryDesc *save_ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
@ 2022-09-19 10:42 a.rybakina <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: a.rybakina @ 2022-09-19 10:42 UTC (permalink / raw)
To: torikoshia <[email protected]>; [email protected]
Hi,
I'm sorry,if this message is duplicated previous this one, but the
previous message is sent incorrectly. I sent it from email address
[email protected].
I liked this idea and after reviewing code I noticed some moments and
I'd rather ask you some questions.
Firstly, I suggest some editing in the comment of commit. I think, it is
turned out the more laconic and the same clear. I wrote it below since I
can't think of any other way to add it.
```
Currently, we have to wait for finishing of the query execution to check
its plan.
This is not so convenient in investigation long-running queries on
production
environments where we cannot use debuggers.
To improve this situation there is proposed the patch containing the
pg_log_query_plan()
function which request to log plan of the specified backend process.
By default, only superusers are allowed to request log of the plan
otherwise
allowing any users to issue this request could create cause lots of log
messages
and it can lead to denial of service.
At the next requesting CHECK_FOR_INTERRUPTS(), the target backend logs
its plan at
LOG_SERVER_ONLY level and therefore this plan will appear in the server
log only,
not to be sent to the client.
```
Secondly, I have question about deleting USE_ASSERT_CHECKING in lock.h?
It supposed to have been checked in another placed of the code by
matching values. I am worry about skipping errors due to untesting with
assert option in the places where it (GetLockMethodLocalHash)
participates and we won't able to get core file in segfault cases. I
might not understand something, then can you please explain to me?
Thirdly, I have incomprehension of the point why save_ActiveQueryDesc is
declared in the pquery.h? I am seemed to save_ActiveQueryDesc to be used
in an once time in the ExecutorRun function and its declaration
superfluous. I added it in the attached patch.
Fourthly, it seems to me there are not enough explanatory comments in
the code. I also added them in the attached patch.
Lastly, I have incomprehension about handling signals since have been
unused it before. Could another signal disabled calling this signal to
log query plan? I noticed this signal to be checked the latest in
procsignal_sigusr1_handler function.
Regards,
--
Alena Rybakina
Postgres Professional
> 19.09.2022, 11:01, "torikoshia" <[email protected]>:
>
> On 2022-05-16 17:02, torikoshia wrote:
>
> On 2022-03-09 19:04, torikoshia wrote:
>
> On 2022-02-08 01:13, Fujii Masao wrote:
>
> AbortSubTransaction() should reset ActiveQueryDesc to
> save_ActiveQueryDesc that ExecutorRun() set, instead
> of NULL?
> Otherwise ActiveQueryDesc of top-level statement will
> be unavailable
> after subtransaction is aborted in the nested statements.
>
>
> I once agreed above suggestion and made v20 patch making
> save_ActiveQueryDesc a global variable, but it caused
> segfault when
> calling pg_log_query_plan() after FreeQueryDesc().
>
> OTOH, doing some kind of reset of ActiveQueryDesc seems
> necessary
> since it also caused segfault when running
> pg_log_query_plan() during
> installcheck.
>
> There may be a better way, but resetting ActiveQueryDesc
> to NULL seems
> safe and simple.
> Of course it makes pg_log_query_plan() useless after a
> subtransaction
> is aborted.
> However, if it does not often happen that people want to
> know the
> running query's plan whose subtransaction is aborted,
> resetting
> ActiveQueryDesc to NULL would be acceptable.
>
> Attached is a patch that sets ActiveQueryDesc to NULL when a
> subtransaction is aborted.
>
> How do you think?
>
> Attached new patch to fix patch apply failures again.
>
> --
> Regards,
>
> --
> Atsushi Torikoshi
> NTT DATA CORPORATION
>
Attachments:
[text/x-patch] diff_query_plan.patch (1.7K, ../../[email protected]/3-diff_query_plan.patch)
download | inline diff:
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index a82ac87457e..65b692b0ddf 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -306,6 +306,12 @@ ExecutorRun(QueryDesc *queryDesc,
{
QueryDesc *save_ActiveQueryDesc;
+ /*
+ * Save value of ActiveQueryDesc before having called
+ * ExecutorRun_hook function due to having reset by
+ * AbortSubTransaction.
+ */
+
save_ActiveQueryDesc = ActiveQueryDesc;
ActiveQueryDesc = queryDesc;
@@ -314,6 +320,7 @@ ExecutorRun(QueryDesc *queryDesc,
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+ /* We set the actual value of ActiveQueryDesc */
ActiveQueryDesc = save_ActiveQueryDesc;
}
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index fc9f9f8e3f0..8e7ce3c976f 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -126,6 +126,7 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+/* Function to handle the signal to output the query plan. */
extern void HandleLogQueryPlanInterrupt(void);
extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 227d24b9d60..d04de1f5566 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -22,7 +22,6 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
-extern PGDLLIMPORT QueryDesc *save_ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
^ permalink raw reply [nested|flat] 124+ messages in thread
* Re: RFC: Logging plan of the running query
@ 2024-01-29 13:02 torikoshia <[email protected]>
0 siblings, 0 replies; 124+ messages in thread
From: torikoshia @ 2024-01-29 13:02 UTC (permalink / raw)
To: [email protected]; +Cc: Étienne BERSAC <[email protected]>; Andres Freund <[email protected]>; [email protected]; [email protected]; [email protected]
Hi,
Updated the patch to fix typos and move
ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction.
BTW since the thread is getting long, I list the some points of the
discussion so far:
# Safety concern
## Catalog access inside CFI
- it seems safe if the CFI call is inside an existing valid
transaction/query state[1]
- We did some tests, for example calling ProcessLogQueryPlanInterrupt()
in every single CHECK_FOR_INTERRUPTS()[2]. This test passed on my env
but was stucked on James's env, so I modified to exit
ProcessLogQueryPlanInterrupt() when target process is inside of lock
acquisition code[3]
## Risk of calling EXPLAIN code in CFI
- EXPLAIN is not a simple logic code, and there exists risk calling it
from CFI. For example, if there is a bug, we may find ourselves in a
situation where we can't cancel the query
- it's a trade-off that's worth making for the introspection benefits
this patch would provide?[4]
# Design
- Although some suggested it should be in auto_explain, current patch
introduces this feature to the core[5]
- When the target query is nested, only the most inner query's plan is
explained. In future, all the nested queries' plans are expected to
explained optionally like auto_explain.log_nested_statements[6]
- When the target process is a parallel worker, the plan is not shown[6]
- When the target query is nested and its subtransaction is aborted,
pg_log_query_plan cannot log the parental query plan after the abort
even parental query is running[7]
- The output corresponds to EXPLAIN with VERBOSE, COST, SETTINGS and
FORMAT text. It doesn't do ANALYZE or show the progress of the query
execution. Future work proposed by Rafael Thofehrn Castro may realize
this[8]
- To prevent assertion error, this patch ensures no page lock is held by
checking all the LocalLock entries before running explain code, but
there is a discussion that ginInsertCleanup() should be modified[9]
It may be not so difficult to improve some of restrictions in "Design",
but I'd like to limit the scope of the 1st patch to make it simpler.
[1]
https://www.postgresql.org/message-id/CAAaqYe9euUZD8bkjXTVcD9e4n5c7kzHzcvuCJXt-xds9X4c7Fw%40mail.gma...
[2]
https://www.postgresql.org/message-id/CAAaqYe8LXVXQhYy3yT0QOHUymdM%3Duha0dJ0%3DBEPzVAx2nG1gsw%40mail...
[3]
https://www.postgresql.org/message-id/0e0e7ca08dff077a625c27a5e0c2ef0a%40oss.nttdata.com
[4]
https://www.postgresql.org/message-id/CAAaqYe8LXVXQhYy3yT0QOHUymdM%3Duha0dJ0%3DBEPzVAx2nG1gsw%40mail...
[5]
https://www.postgresql.org/message-id/CAAaqYe_1EuoTudAz8mr8-qtN5SoNtYRm4JM2J9CqeverpE3B2A%40mail.gma...
[6]
https://www.postgresql.org/message-id/CAExHW5sh4ahrJgmMAGfptWVmESt1JLKCNm148XVxTunRr%2B-6gA%40mail.g...
[7]
https://www.postgresql.org/message-id/3d121ed5f81cef588bac836b43f5d1f9%40oss.nttdata.com
[8]
https://www.postgresql.org/message-id/c161b5e7e1888eb9c9eb182a7d9dcf89%40oss.nttdata.com
[9]
https://www.postgresql.org/message-id/20220201.172757.1480996662235658750.horikyota.ntt%40gmail.com
--
Regards,
--
Atsushi Torikoshi
NTT DATA Group Corporation
Attachments:
[text/x-diff] v35-0001-Add-function-to-log-the-plan-of-the-query.patch (30.1K, ../../[email protected]/2-v35-0001-Add-function-to-log-the-plan-of-the-query.patch)
download | inline diff:
From 65786ad6c2a9b656c3fd36a45118a39a66da0236 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Mon, 29 Jan 2024 21:40:04 +0900
Subject: [PATCH v35] Add function to log the plan of the query
Currently, we have to wait for the query execution to finish
to check its plan. This is not so convenient when
investigating long-running queries on production environments
where we cannot use debuggers.
To improve this situation, this patch adds
pg_log_query_plan() function that requests to log the
plan of the specified backend process.
By default, only superusers are allowed to request to log the
plans because allowing any users to issue this request at an
unbounded rate would cause lots of log messages and which can
lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its plan at LOG_SERVER_ONLY level, so
that these plans will appear in the server log but not be sent
to the client.
Reviewed-by: Bharath Rupireddy, Fujii Masao, Dilip Kumar,
Masahiro Ikeda, Ekaterina Sokolova, Justin Pryzby, Kyotaro
Horiguchi, Robert Treat, Alena Rybakina, Ashutosh Bapat
Co-authored-by: James Coleman <[email protected]>
---
contrib/auto_explain/auto_explain.c | 23 +-
doc/src/sgml/func.sgml | 50 +++++
src/backend/access/transam/xact.c | 17 ++
src/backend/catalog/system_functions.sql | 2 +
src/backend/commands/explain.c | 208 ++++++++++++++++++-
src/backend/executor/execMain.c | 14 ++
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/storage/lmgr/lock.c | 9 +-
src/backend/tcop/postgres.c | 4 +
src/backend/utils/init/globals.c | 2 +
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/explain.h | 9 +
src/include/miscadmin.h | 1 +
src/include/storage/lock.h | 2 -
src/include/storage/procsignal.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/elog.h | 1 +
src/test/regress/expected/misc_functions.out | 54 ++++-
src/test/regress/sql/misc_functions.sql | 41 +++-
19 files changed, 402 insertions(+), 48 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c7aacd7812..c0f2ca4c18 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -399,26 +399,9 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->format = auto_explain_log_format;
es->settings = auto_explain_log_settings;
- ExplainBeginOutput(es);
- ExplainQueryText(es, queryDesc);
- ExplainQueryParameters(es, queryDesc->params, auto_explain_log_parameter_max_length);
- ExplainPrintPlan(es, queryDesc);
- if (es->analyze && auto_explain_log_triggers)
- ExplainPrintTriggers(es, queryDesc);
- if (es->costs)
- ExplainPrintJITSummary(es, queryDesc);
- ExplainEndOutput(es);
-
- /* Remove last line break */
- if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
- es->str->data[--es->str->len] = '\0';
-
- /* Fix JSON to output an object */
- if (auto_explain_log_format == EXPLAIN_FORMAT_JSON)
- {
- es->str->data[0] = '{';
- es->str->data[es->str->len - 1] = '}';
- }
+ ExplainAssembleLogOutput(es, queryDesc, auto_explain_log_format,
+ auto_explain_log_triggers,
+ auto_explain_log_parameter_max_length);
/*
* Note: we rely on the existing logging of context or
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6788ba8ef4..48e8748fd1 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26524,6 +26524,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_query_plan</primary>
+ </indexterm>
+ <function>pg_log_query_plan</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the plan of the query currently running on the
+ backend with specified process ID.
+ It will be logged at <literal>LOG</literal> message level and
+ will appear in the server log based on the log
+ configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client
+ regardless of <xref linkend="guc-client-min-messages"/>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -27300,6 +27319,37 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_query_plan</function> can be used
+ to log the plan of a backend process. For example:
+<programlisting>
+postgres=# SELECT pg_log_query_plan(201116);
+ pg_log_query_plan
+---------------------------
+ t
+(1 row)
+</programlisting>
+The format of the query plan is the same as when <literal>VERBOSE</literal>,
+<literal>COSTS</literal>, <literal>SETTINGS</literal> and
+<literal>FORMAT TEXT</literal> are used in the <command>EXPLAIN</command>
+command. For example:
+<screen>
+LOG: query plan running on backend with PID 201116 is:
+ Query Text: SELECT * FROM pgbench_accounts;
+ Seq Scan on public.pgbench_accounts (cost=0.00..52787.00 rows=2000000 width=97)
+ Output: aid, bid, abalance, filler
+ Settings: work_mem = '1MB'
+ Query Identifier: 8621255546560739680
+</screen>
+ Note that when statements are executed inside a function, only the
+ plan of the most deeply nested query is logged.
+ </para>
+
+ <para>
+ When a subtransaction is aborted, <function>pg_log_query_plan</function>
+ cannot log the query plan after the abort.
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 464858117e..d0b5954627 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -61,6 +61,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "tcop/pquery.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/combocid.h"
@@ -2750,6 +2751,14 @@ AbortTransaction(void)
*/
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
+ /*
+ * Reset pg_log_query_plan() related global variables.
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+ ProcessLogQueryPlanInterruptActive = false;
+
/*
* check the current transaction state
*/
@@ -5109,6 +5118,14 @@ AbortSubTransaction(void)
*/
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
+ /*
+ * Reset pg_log_query_plan() related global variables.
+ * When ActiveQueryDesc is referenced after abort, some of its elements
+ * are freed. To avoid accessing them, reset ActiveQueryDesc here.
+ */
+ ActiveQueryDesc = NULL;
+ ProcessLogQueryPlanInterruptActive = false;
+
/*
* check the current transaction state
*/
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 346cfb98a0..c64ca9946e 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -757,6 +757,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_logicalmapdir() FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_ls_replslotdir(text) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 843472e6dd..6d8d2c2f97 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -21,6 +21,7 @@
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
+#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -28,7 +29,10 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "utils/backend_status.h"
#include "utils/builtins.h"
#include "utils/guc_tables.h"
#include "utils/json.h"
@@ -40,6 +44,7 @@
#include "utils/typcache.h"
#include "utils/xml.h"
+bool ProcessLogQueryPlanInterruptActive = false;
/* Hook for plugins to get control in ExplainOneQuery() */
ExplainOneQuery_hook_type ExplainOneQuery_hook = NULL;
@@ -737,6 +742,37 @@ ExplainPrintSettings(ExplainState *es)
}
}
+/*
+ * ExplainAssembleLogOutput -
+ * Assemble es->str for logging according to specified contents and format
+ */
+
+void
+ExplainAssembleLogOutput(ExplainState *es, QueryDesc *queryDesc, int logFormat,
+ bool logTriggers, int logParameterMaxLength)
+{
+ ExplainBeginOutput(es);
+ ExplainQueryText(es, queryDesc);
+ ExplainQueryParameters(es, queryDesc->params, logParameterMaxLength);
+ ExplainPrintPlan(es, queryDesc);
+ if (es->analyze && logTriggers)
+ ExplainPrintTriggers(es, queryDesc);
+ if (es->costs)
+ ExplainPrintJITSummary(es, queryDesc);
+ ExplainEndOutput(es);
+
+ /* Remove last line break */
+ if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+ es->str->data[--es->str->len] = '\0';
+
+ /* Fix JSON to output an object */
+ if (logFormat == EXPLAIN_FORMAT_JSON)
+ {
+ es->str->data[0] = '{';
+ es->str->data[es->str->len - 1] = '}';
+ }
+}
+
/*
* ExplainPrintPlan -
* convert a QueryDesc's plan tree to text and append it to es->str
@@ -1647,6 +1683,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
/*
* We have to forcibly clean up the instrumentation state because we
* haven't done ExecutorEnd yet. This is pretty grotty ...
+ * This cleanup should not be done when the query has already been
+ * executed and explain has been called by signal, as the target query
+ * may use instrumentation and clean itself up.
*
* Note: contrib/auto_explain could cause instrumentation to be set up
* even though we didn't ask for it here. Be careful not to print any
@@ -1654,7 +1693,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
* InstrEndLoop call anyway, if possible, to reduce the number of cases
* auto_explain has to contend with.
*/
- if (planstate->instrument)
+ if (planstate->instrument && !es->signaled)
InstrEndLoop(planstate->instrument);
if (es->analyze &&
@@ -5082,3 +5121,170 @@ escape_yaml(StringInfo buf, const char *str)
{
escape_json(buf, str);
}
+
+/*
+ * HandleLogQueryPlanInterrupt
+ * Handle receipt of an interrupt indicating logging the plan of
+ * the currently running query.
+ *
+ * All the actual work is deferred to ProcessLogQueryPlanInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+void
+HandleLogQueryPlanInterrupt(void)
+{
+ InterruptPending = true;
+ LogQueryPlanPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogQueryPlanInterrupt
+ * Perform logging the plan of the currently running query on this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange to call
+ * this function if we see LogQueryPlanPending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because the target
+ * process for logging plan is a backend.
+ */
+void
+ProcessLogQueryPlanInterrupt(void)
+{
+ ExplainState *es;
+ HASH_SEQ_STATUS status;
+ LOCALLOCK *locallock;
+ MemoryContext cxt;
+ MemoryContext old_cxt;
+ LogQueryPlanPending = false;
+
+ /* Cannot re-enter. */
+ if (ProcessLogQueryPlanInterruptActive)
+ return;
+
+ ProcessLogQueryPlanInterruptActive = true;
+
+ if (ActiveQueryDesc == NULL)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("backend with PID %d is not running a query or a subtransaction is aborted",
+ MyProcPid),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+
+ /*
+ * Ensure no page lock is held on this process.
+ *
+ * If page lock is held at the time of the interrupt, we can't acquire any
+ * other heavyweight lock, which might be necessary for explaining the plan
+ * when retrieving column names.
+ *
+ * This may be overkill, but since page locks are held for a short duration
+ * we check all the LocalLock entries and when finding even one, give up
+ * logging the plan.
+ */
+ hash_seq_init(&status, GetLockMethodLocalHash());
+ while ((locallock = (LOCALLOCK *) hash_seq_search(&status)) != NULL)
+ {
+ if (LOCALLOCK_LOCKTAG(*locallock) == LOCKTAG_PAGE)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to page lock conflicts"),
+ errdetail("You can try again in a moment."));
+ hash_seq_term(&status);
+
+ ProcessLogQueryPlanInterruptActive = false;
+ return;
+ }
+ }
+
+ /*
+ * Ensure no lock is already held on the lockable object.
+ * Otherwise EXPLAIN can be also hold on it.
+ */
+ if (MyProc->heldLocks)
+ {
+ ereport(LOG_SERVER_ONLY,
+ errmsg("ignored request for logging query plan due to lock conflicts"),
+ errdetail("You can try again in a moment."));
+ return;
+ }
+
+ cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "log_query_plan temporary context",
+ ALLOCSET_DEFAULT_SIZES);
+
+ old_cxt = MemoryContextSwitchTo(cxt);
+
+ es = NewExplainState();
+
+ es->format = EXPLAIN_FORMAT_TEXT;
+ es->settings = true;
+ es->verbose = true;
+ es->signaled = true;
+
+ ExplainAssembleLogOutput(es, ActiveQueryDesc, EXPLAIN_FORMAT_TEXT, 0, -1);
+
+ ereport(LOG_SERVER_ONLY,
+ errmsg("query plan running on backend with PID %d is:\n%s",
+ MyProcPid, es->str->data),
+ errhidestmt(true),
+ errhidecontext(true));
+
+ MemoryContextSwitchTo(old_cxt);
+ MemoryContextDelete(cxt);
+
+ ProcessLogQueryPlanInterruptActive = false;
+}
+
+/*
+ * pg_log_query_plan
+ * Signal a backend process to log the query plan of the running query.
+ *
+ * By default, only superusers are allowed to signal to log the plan because
+ * allowing any users to issue this request at an unbounded rate would
+ * cause lots of log messages and which can lead to denial of service.
+ * Additional roles can be permitted with GRANT.
+ */
+Datum
+pg_log_query_plan(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+ PgBackendStatus *be_status;
+
+ proc = BackendPidGetProc(pid);
+
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ be_status = pgstat_get_beentry_by_backend_id(proc->backendId);
+ if (be_status->st_backendType != B_BACKEND)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL client backend process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ if (SendProcSignal(pid, PROCSIG_LOG_QUERY_PLAN, proc->backendId) < 0)
+ {
+ /* Again, just a warning to allow loops */
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 13a9b7da83..4c7fe35f8f 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -78,6 +78,9 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
/* Hook for plugin to get control in ExecCheckPermissions() */
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
+/* Currently executing query's QueryDesc */
+QueryDesc *ActiveQueryDesc = NULL;
+
/* decls for local routines only used within this module */
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
@@ -303,10 +306,21 @@ ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count,
bool execute_once)
{
+ /*
+ * Update ActiveQueryDesc here to enable retrieval of the currently
+ * running queryDesc for nested queries.
+ */
+ QueryDesc *save_ActiveQueryDesc;
+
+ save_ActiveQueryDesc = ActiveQueryDesc;
+ ActiveQueryDesc = queryDesc;
+
if (ExecutorRun_hook)
(*ExecutorRun_hook) (queryDesc, direction, count, execute_once);
else
standard_ExecutorRun(queryDesc, direction, count, execute_once);
+
+ ActiveQueryDesc = save_ActiveQueryDesc;
}
void
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index e84619e5a5..69638e64bb 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -20,6 +20,7 @@
#include "access/parallel.h"
#include "port/pg_bitutils.h"
#include "commands/async.h"
+#include "commands/explain.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "replication/logicalworker.h"
@@ -658,6 +659,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_QUERY_PLAN))
+ HandleLogQueryPlanInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c70a1adb9a..fed2ea1924 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -602,17 +602,18 @@ LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode)
return (locallock && locallock->nLocks > 0);
}
-#ifdef USE_ASSERT_CHECKING
/*
- * GetLockMethodLocalHash -- return the hash of local locks, for modules that
- * evaluate assertions based on all locks held.
+ * GetLockMethodLocalHash -- return the hash of local locks, mainly for
+ * modules that evaluate assertions based on all locks held.
+ *
+ * NOTE: When there are many entries in LockMethodLocalHash, calling this
+ * function and looking into all of them can lead to performance problems.
*/
HTAB *
GetLockMethodLocalHash(void)
{
return LockMethodLocalHash;
}
-#endif
/*
* LockHasWaiters -- look up 'locktag' and check if releasing this
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 1a34bd3715..07797ef7dd 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -37,6 +37,7 @@
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/event_trigger.h"
+#include "commands/explain.h"
#include "commands/prepare.h"
#include "common/pg_prng.h"
#include "jit/jit.h"
@@ -3457,6 +3458,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (LogQueryPlanPending)
+ ProcessLogQueryPlanInterrupt();
+
if (ParallelApplyMessagePending)
HandleParallelApplyMessages();
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 88b03e8fa3..063a92a5bc 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -37,6 +37,8 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
+volatile sig_atomic_t LogQueryPlanPending = false;
+
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 29af4ce65d..26b11d6a12 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8274,6 +8274,12 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# logging plan of the running query on the specified backend
+{ oid => '8000', descr => 'log plan of the running query on the specified backend',
+ proname => 'pg_log_query_plan',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_query_plan' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 1b44d483d6..bbca106e92 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -17,6 +17,8 @@
#include "lib/stringinfo.h"
#include "parser/parse_node.h"
+extern PGDLLIMPORT bool ProcessLogQueryPlanInterruptActive;
+
typedef enum ExplainFormat
{
EXPLAIN_FORMAT_TEXT,
@@ -60,6 +62,7 @@ typedef struct ExplainState
bool hide_workers; /* set if we find an invisible Gather */
/* state related to the current plan node */
ExplainWorkersState *workers_state; /* needed if parallel plan */
+ bool signaled; /* whether explain is called by signal */
} ExplainState;
/* Hook for plugins to get control in ExplainOneQuery() */
@@ -94,6 +97,10 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
const instr_time *planduration,
const BufferUsage *bufusage);
+extern void ExplainAssembleLogOutput(ExplainState *es, QueryDesc *queryDesc,
+ int logFormat, bool logTriggers,
+ int logParameterMaxLength);
+
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
@@ -126,4 +133,6 @@ extern void ExplainOpenGroup(const char *objtype, const char *labelname,
extern void ExplainCloseGroup(const char *objtype, const char *labelname,
bool labeled, ExplainState *es);
+extern void HandleLogQueryPlanInterrupt(void);
+extern void ProcessLogQueryPlanInterrupt(void);
#endif /* EXPLAIN_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0b01c1f093..3914ba9f71 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -95,6 +95,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogQueryPlanPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 00679624f7..382930073a 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -569,9 +569,7 @@ extern void LockReleaseSession(LOCKMETHODID lockmethodid);
extern void LockReleaseCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern void LockReassignCurrentOwner(LOCALLOCK **locallocks, int nlocks);
extern bool LockHeldByMe(const LOCKTAG *locktag, LOCKMODE lockmode);
-#ifdef USE_ASSERT_CHECKING
extern HTAB *GetLockMethodLocalHash(void);
-#endif
extern bool LockHasWaiters(const LOCKTAG *locktag,
LOCKMODE lockmode, bool sessionLock);
extern VirtualTransactionId *GetLockConflicts(const LOCKTAG *locktag,
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 52dcb4c2ad..400c527292 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_QUERY_PLAN, /* ask backend to log plan of the current query */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index 073fb323bc..3dd7edf93c 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -21,7 +21,7 @@ struct PlannedStmt; /* avoid including plannodes.h here */
extern PGDLLIMPORT Portal ActivePortal;
-
+extern PGDLLIMPORT QueryDesc *ActiveQueryDesc;
extern PortalStrategy ChoosePortalStrategy(List *stmts);
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 761ee2512d..3d5b4b37fd 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -167,6 +167,7 @@ struct Node;
extern bool message_level_is_interesting(int elevel);
+extern PGDLLIMPORT bool ProcessLogQueryPlanInterruptActive;
extern bool errstart(int elevel, const char *domain);
extern pg_attribute_cold bool errstart_cold(int elevel, const char *domain);
extern void errfinish(const char *filename, int lineno, const char *funcname);
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 7c15477104..f34f2bcd7b 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -276,14 +276,15 @@ SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
../jkl/mno
(1 row)
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -297,8 +298,8 @@ SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
t
(1 row)
-CREATE ROLE regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+CREATE ROLE regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
has_function_privilege
------------------------
@@ -306,15 +307,15 @@ SELECT has_function_privilege('regress_log_memory',
(1 row)
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
-SELECT has_function_privilege('regress_log_memory',
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
has_function_privilege
------------------------
t
(1 row)
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
pg_log_backend_memory_contexts
--------------------------------
@@ -323,8 +324,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
-DROP ROLE regress_log_memory;
+ FROM regress_log_function;
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+ pg_log_query_plan
+-------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 851dad90f4..f472a839bf 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -68,39 +68,60 @@ SELECT test_canonicalize_path('./abc/./def/.');
SELECT test_canonicalize_path('./abc/././def/.');
SELECT test_canonicalize_path('./abc/./def/.././ghi/../../../jkl/mno');
+-- Test logging functions
--
--- pg_log_backend_memory_contexts()
---
--- Memory contexts are logged and they are not returned to the function.
+-- The outputs of these functions are logged and they are not returned to
+-- the function.
-- Furthermore, their contents can vary depending on the timing. However,
-- we can at least verify that the code doesn't fail, and that the
-- permissions are set properly.
--
+-- pg_log_backend_memory_contexts()
+
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity
WHERE backend_type = 'checkpointer';
-CREATE ROLE regress_log_memory;
+CREATE ROLE regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
GRANT EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- TO regress_log_memory;
+ TO regress_log_function;
-SELECT has_function_privilege('regress_log_memory',
+SELECT has_function_privilege('regress_log_function',
'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- yes
-SET ROLE regress_log_memory;
+SET ROLE regress_log_function;
SELECT pg_log_backend_memory_contexts(pg_backend_pid());
RESET ROLE;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer)
- FROM regress_log_memory;
+ FROM regress_log_function;
+
+-- pg_log_query_plan()
+SELECT pg_log_query_plan(pg_backend_pid());
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ TO regress_log_function;
+
+SELECT has_function_privilege('regress_log_function',
+ 'pg_log_query_plan(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_function;
+SELECT pg_log_query_plan(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_query_plan(integer)
+ FROM regress_log_function;
-DROP ROLE regress_log_memory;
+DROP ROLE regress_log_function;
--
-- Test some built-in SRFs
base-commit: 6a1ea02c491d16474a6214603dce40b5b122d4d1
--
2.39.2
^ permalink raw reply [nested|flat] 124+ messages in thread
end of thread, other threads:[~2026-07-07 14:46 UTC | newest]
Thread overview: 124+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-19 06:11 [PATCH v45 4/7] Shared-memory based stats collector Kyotaro Horiguchi <[email protected]>
2021-05-12 11:24 RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-12 12:07 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-05-12 12:33 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-12 16:08 ` Re: RFC: Logging plan of the running query Laurenz Albe <[email protected]>
2021-05-13 08:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 09:38 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-12 12:55 ` Re: RFC: Logging plan of the running query Matthias van de Meent <[email protected]>
2021-05-13 08:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-12 14:40 ` Re: RFC: Logging plan of the running query Julien Rouhaud <[email protected]>
2021-05-13 09:13 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 09:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:36 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 09:49 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 10:46 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:43 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 11:45 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-05-13 11:48 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-13 12:57 ` Re: RFC: Logging plan of the running query Dilip Kumar <[email protected]>
2021-05-28 06:51 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 07:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-09 14:04 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-06-10 02:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-10 16:20 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-14 12:18 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-15 04:27 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-06-16 11:36 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-06-22 02:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-01 06:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-02 14:21 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-07-09 05:05 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-13 14:11 ` Re: RFC: Logging plan of the running query Masahiro Ikeda <[email protected]>
2021-07-19 02:28 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-07-19 06:07 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:34 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-07-27 18:45 ` Re: RFC: Logging plan of the running query Pavel Stehule <[email protected]>
2021-07-28 11:44 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 12:22 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-10 15:21 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-08-11 12:14 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-08-19 16:12 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2021-09-07 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-09-08 12:06 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-13 14:28 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-10-15 06:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-10-15 10:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-12 18:37 ` Re: RFC: Logging plan of the running query Justin Pryzby <[email protected]>
2021-11-15 14:00 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-13 13:29 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-11-15 12:59 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-15 14:15 ` Re: RFC: Logging plan of the running query Bharath Rupireddy <[email protected]>
2021-11-16 11:48 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-05-13 10:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-02 11:32 Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-03 12:48 ` Re: RFC: Logging plan of the running query Daniel Gustafsson <[email protected]>
2021-11-04 12:49 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2021-11-17 13:44 ` Re: RFC: Logging plan of the running query Ekaterina Sokolova <[email protected]>
2021-11-26 03:39 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-08 15:42 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-03-10 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-21 12:40 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-31 18:51 ` Re: RFC: Logging plan of the running query Sadeq Dousti <[email protected]>
2025-04-03 05:34 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-03-31 19:24 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-04-03 05:59 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-04-05 06:13 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-04-27 23:55 ` Re: RFC: Logging plan of the running query Hannu Krosing <[email protected]>
2025-05-20 13:17 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-05-22 15:07 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-05-23 08:50 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-06-02 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-01 13:35 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-09 17:59 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-16 13:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-16 15:05 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-09-19 12:42 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-09-19 17:03 ` Re: RFC: Logging plan of the running query Rafael Thofehrn Castro <[email protected]>
2025-09-20 03:21 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2025-09-24 16:34 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-01 09:11 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-10-16 20:15 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-10-20 12:15 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-18 20:19 ` Re: RFC: Logging plan of the running query Akshat Jaimini <[email protected]>
2025-11-19 03:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-19 17:23 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-20 01:52 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2025-11-20 13:17 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2025-11-25 12:43 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-08 06:08 ` Re: RFC: Logging plan of the running query Lukas Fittl <[email protected]>
2026-06-08 11:48 ` Re: RFC: Logging plan of the running query Atsushi Torikoshi <[email protected]>
2026-06-15 13:26 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-22 13:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-24 12:56 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-24 14:35 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-25 09:43 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-25 10:33 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-06-26 12:50 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-06-29 04:23 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-06-29 16:30 ` Re: RFC: Logging plan of the running query Robert Haas <[email protected]>
2026-07-01 10:42 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-01 13:09 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-02 11:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-06 06:21 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2026-07-06 13:05 ` Re: RFC: Logging plan of the running query Andrei Lepikhov <[email protected]>
2026-07-07 14:46 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-08 08:18 Re: RFC: Logging plan of the running query Kyotaro Horiguchi <[email protected]>
2022-02-08 15:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-02-21 17:39 ` Re: RFC: Logging plan of the running query Fujii Masao <[email protected]>
2022-03-09 10:04 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-05-16 08:02 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-08 10:19 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-19 08:47 ` Re: RFC: Logging plan of the running query Алена Рыбакина <[email protected]>
2022-09-20 14:41 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-21 08:30 ` Re: RFC: Logging plan of the running query Alena Rybakina <[email protected]>
2022-09-22 14:12 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2022-09-23 09:43 ` Re: RFC: Logging plan of the running query Alena Rybakina <[email protected]>
2022-12-06 18:41 ` Re: RFC: Logging plan of the running query Andres Freund <[email protected]>
2022-12-08 05:10 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2023-06-02 17:51 ` Re: RFC: Logging plan of the running query James Coleman <[email protected]>
2023-06-05 08:30 ` Re: RFC: Logging plan of the running query torikoshia <[email protected]>
2023-06-05 18:26 ` Re: RFC: Logging plan of the running query James Coleman <[email protected]>
2022-09-16 18:51 RFC: Logging plan of the running query a.rybakina <[email protected]>
2022-09-19 10:42 Re: RFC: Logging plan of the running query a.rybakina <[email protected]>
2024-01-29 13:02 Re: RFC: Logging plan of the running query torikoshia <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox