Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1j2Rx2-0000KS-Fp for pgsql-hackers@arkaria.postgresql.org; Fri, 14 Feb 2020 03:47:36 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1j2Rx1-0007WN-8b for pgsql-hackers@arkaria.postgresql.org; Fri, 14 Feb 2020 03:47:35 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1j2Rx0-0007W9-Tx for pgsql-hackers@lists.postgresql.org; Fri, 14 Feb 2020 03:47:35 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1j2Rws-0004fi-E3 for pgsql-hackers@postgresql.org; Fri, 14 Feb 2020 03:47:34 +0000 Received: from hnk.local (sp49-97-94-18.msc.spmode.ne.jp [49.97.94.18]) by oss.nttdata.com (Postfix) with ESMTPSA id 0434B603C2; Fri, 14 Feb 2020 12:47:19 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.101.5 at oss.nttdata.com Subject: Re: Wait event that should be reported while waiting for WAL archiving to finish To: Michael Paquier Cc: pgsql-hackers@postgresql.org References: <20200213032804.GB1520@paquier.xyz> <949931aa-4ed4-d867-a7b5-de9c02b2292b@oss.nttdata.com> <20200213073035.GI1520@paquier.xyz> From: Fujii Masao Message-ID: Date: Fri, 14 Feb 2020 12:47:19 +0900 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: <20200213073035.GI1520@paquier.xyz> Content-Type: multipart/mixed; boundary="------------1EE62B1B9D75C9C1CEE71C7A" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------1EE62B1B9D75C9C1CEE71C7A Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 2020/02/13 16:30, Michael Paquier wrote: > On Thu, Feb 13, 2020 at 03:35:50PM +0900, Fujii Masao wrote: >> I found that the wait events "LogicalRewriteTruncate" and >> "GSSOpenServer" are not documented. I'm thinking to add >> them into doc separately if ok. > > Nice catch. The ordering of the entries is not respected either for > GSSOpenServer in pgstat.h. The portion for the code and the docs can > be fixed in back-branches, but not the enum list in WaitEventClient or > we would have an ABI breakage. But this can be fixed on HEAD. Can > you take care of it? Yes. Patch attached. logical_rewrite_truncate_v1.patch adds the description of LogicalRewriteTruncate into the doc. This needs to be back-patched to v10 where commit 249cf070e3 introduced LogicalRewriteTruncate event. gss_open_server_v1.patch adds the description of GSSOpenServer into the doc and update the code in pgstat_get_wait_client(). This needs to be applied in v12 where commit b0b39f72b9 introduced GSSOpenServer event. gss_open_server_for_master_v1.patch does not only what the above patch does but also update wait event enum into alphabetical order. This needs to be applied in the master. > >> SyncRep >> Waiting for confirmation from remote server during synchronous replication. >> >> + >> + BackupWaitWalArchive >> + Waiting for WAL files required for the backup to be successfully archived. >> + > > The category IPC is adapted. You forgot to update the markup morerows > from "36" to "37", causing the table of the wait events to have a > weird format (the bottom should be incorrect). Fixed. Thanks for the review! > >> + pgstat_report_wait_start(WAIT_EVENT_BACKUP_WAIT_WAL_ARCHIVE); >> while (XLogArchiveIsBusy(lastxlogfilename) || >> XLogArchiveIsBusy(histfilename)) >> { >> @@ -11120,6 +11121,7 @@ do_pg_stop_backup(char *labelfile, bool waitforarchive, TimeLineID *stoptli_p) >> "but the database backup will not be usable without all the WAL segments."))); >> } >> } >> + pgstat_report_wait_end(); > > Okay, that position is right. > >> @@ -3848,6 +3848,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) >> case WAIT_EVENT_SYNC_REP: >> event_name = "SyncRep"; >> break; >> + case WAIT_EVENT_BACKUP_WAIT_WAL_ARCHIVE: >> + event_name = "BackupWaitWalArchive"; >> + break; >> /* no default case, so that compiler will warn */ >> [...] >> @@ -853,7 +853,8 @@ typedef enum >> WAIT_EVENT_REPLICATION_ORIGIN_DROP, >> WAIT_EVENT_REPLICATION_SLOT_DROP, >> WAIT_EVENT_SAFE_SNAPSHOT, >> - WAIT_EVENT_SYNC_REP >> + WAIT_EVENT_SYNC_REP, >> + WAIT_EVENT_BACKUP_WAIT_WAL_ARCHIVE >> } WaitEventIPC; > > It would be good to keep entries in alphabetical order in the header, > the code and in the docs (see the effort from 5ef037c), and your patch > is missing that concept for all three places where it matters for this > new event. Fixed. Patch attached. Regards, -- Fujii Masao NTT DATA CORPORATION Advanced Platform Technology Group Research and Development Headquarters --------------1EE62B1B9D75C9C1CEE71C7A Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="gss_open_server_for_master_v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="gss_open_server_for_master_v1.patch" diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 9129f79bbf..027df985ac 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1293,7 +1293,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser Waiting in main loop of WAL writer process. - Client + Client ClientRead Waiting to read data from the client. @@ -1301,6 +1301,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser ClientWrite Waiting to write data to the client. + + GSSOpenServer + Waiting to read data from the client while establishing the GSSAPI session. + LibPQWalReceiverConnect Waiting in WAL receiver to establish connection to remote server. diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c index 7169509a79..59dc4f31ab 100644 --- a/src/backend/postmaster/pgstat.c +++ b/src/backend/postmaster/pgstat.c @@ -3697,6 +3697,9 @@ pgstat_get_wait_client(WaitEventClient w) case WAIT_EVENT_CLIENT_WRITE: event_name = "ClientWrite"; break; + case WAIT_EVENT_GSS_OPEN_SERVER: + event_name = "GSSOpenServer"; + break; case WAIT_EVENT_LIBPQWALRECEIVER_CONNECT: event_name = "LibPQWalReceiverConnect"; break; @@ -3715,9 +3718,6 @@ pgstat_get_wait_client(WaitEventClient w) case WAIT_EVENT_WAL_SENDER_WRITE_DATA: event_name = "WalSenderWriteData"; break; - case WAIT_EVENT_GSS_OPEN_SERVER: - event_name = "GSSOpenServer"; - break; /* no default case, so that compiler will warn */ } diff --git a/src/include/pgstat.h b/src/include/pgstat.h index aecb6013f0..3a65a51696 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -799,13 +799,13 @@ typedef enum { WAIT_EVENT_CLIENT_READ = PG_WAIT_CLIENT, WAIT_EVENT_CLIENT_WRITE, + WAIT_EVENT_GSS_OPEN_SERVER, WAIT_EVENT_LIBPQWALRECEIVER_CONNECT, WAIT_EVENT_LIBPQWALRECEIVER_RECEIVE, WAIT_EVENT_SSL_OPEN_SERVER, WAIT_EVENT_WAL_RECEIVER_WAIT_START, WAIT_EVENT_WAL_SENDER_WAIT_WAL, WAIT_EVENT_WAL_SENDER_WRITE_DATA, - WAIT_EVENT_GSS_OPEN_SERVER, } WaitEventClient; /* ---------- --------------1EE62B1B9D75C9C1CEE71C7A Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="gss_open_server_v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="gss_open_server_v1.patch" diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 9129f79bbf..027df985ac 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1293,7 +1293,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser Waiting in main loop of WAL writer process. - Client + Client ClientRead Waiting to read data from the client. @@ -1301,6 +1301,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser ClientWrite Waiting to write data to the client. + + GSSOpenServer + Waiting to read data from the client while establishing the GSSAPI session. + LibPQWalReceiverConnect Waiting in WAL receiver to establish connection to remote server. diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c index 7169509a79..59dc4f31ab 100644 --- a/src/backend/postmaster/pgstat.c +++ b/src/backend/postmaster/pgstat.c @@ -3697,6 +3697,9 @@ pgstat_get_wait_client(WaitEventClient w) case WAIT_EVENT_CLIENT_WRITE: event_name = "ClientWrite"; break; + case WAIT_EVENT_GSS_OPEN_SERVER: + event_name = "GSSOpenServer"; + break; case WAIT_EVENT_LIBPQWALRECEIVER_CONNECT: event_name = "LibPQWalReceiverConnect"; break; @@ -3715,9 +3718,6 @@ pgstat_get_wait_client(WaitEventClient w) case WAIT_EVENT_WAL_SENDER_WRITE_DATA: event_name = "WalSenderWriteData"; break; - case WAIT_EVENT_GSS_OPEN_SERVER: - event_name = "GSSOpenServer"; - break; /* no default case, so that compiler will warn */ } --------------1EE62B1B9D75C9C1CEE71C7A Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="wait_event_wal_archive_v2.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="wait_event_wal_archive_v2.patch" diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 9129f79bbf..411076aaf9 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1331,7 +1331,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser Waiting in an extension. - IPC + IPC + BackupWaitWalArchive + Waiting for WAL files required for the backup to be successfully archived. + + BgWorkerShutdown Waiting for background worker to shut down. diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 3813eadfb4..d79417b443 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -11095,6 +11095,7 @@ do_pg_stop_backup(char *labelfile, bool waitforarchive, TimeLineID *stoptli_p) seconds_before_warning = 60; waits = 0; + pgstat_report_wait_start(WAIT_EVENT_BACKUP_WAIT_WAL_ARCHIVE); while (XLogArchiveIsBusy(lastxlogfilename) || XLogArchiveIsBusy(histfilename)) { @@ -11120,6 +11121,7 @@ do_pg_stop_backup(char *labelfile, bool waitforarchive, TimeLineID *stoptli_p) "but the database backup will not be usable without all the WAL segments."))); } } + pgstat_report_wait_end(); ereport(NOTICE, (errmsg("all required WAL segments have been archived"))); diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c index 7169509a79..0b2c7ee916 100644 --- a/src/backend/postmaster/pgstat.c +++ b/src/backend/postmaster/pgstat.c @@ -3737,6 +3737,9 @@ pgstat_get_wait_ipc(WaitEventIPC w) switch (w) { + case WAIT_EVENT_BACKUP_WAIT_WAL_ARCHIVE: + event_name = "BackupWaitWalArchive"; + break; case WAIT_EVENT_BGWORKER_SHUTDOWN: event_name = "BgWorkerShutdown"; break; diff --git a/src/include/pgstat.h b/src/include/pgstat.h index aecb6013f0..348dcdc21f 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -817,7 +817,8 @@ typedef enum */ typedef enum { - WAIT_EVENT_BGWORKER_SHUTDOWN = PG_WAIT_IPC, + WAIT_EVENT_BACKUP_WAIT_WAL_ARCHIVE = PG_WAIT_IPC, + WAIT_EVENT_BGWORKER_SHUTDOWN, WAIT_EVENT_BGWORKER_STARTUP, WAIT_EVENT_BTREE_PAGE, WAIT_EVENT_CLOG_GROUP_UPDATE, --------------1EE62B1B9D75C9C1CEE71C7A Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="logical_rewrite_truncate_v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="logical_rewrite_truncate_v1.patch" diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 9129f79bbf..0e5a4a0b86 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1493,7 +1493,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser Waiting to apply WAL at recovery because it is delayed. - IO + IO BufFileRead Waiting for a read from a buffered file. @@ -1609,6 +1609,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser LogicalRewriteSync Waiting for logical rewrite mappings to reach stable storage. + + LogicalRewriteTruncate + Waiting for truncate of mapping data during a logical rewrite. + LogicalRewriteWrite Waiting for a write of logical rewrite mappings. --------------1EE62B1B9D75C9C1CEE71C7A--