Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jfovw-0008Vq-K5 for pgsql-admin@arkaria.postgresql.org; Mon, 01 Jun 2020 18:13:12 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jfovv-0000I4-Ha for pgsql-admin@arkaria.postgresql.org; Mon, 01 Jun 2020 18:13:11 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jfovv-0000Hw-8G for pgsql-admin@lists.postgresql.org; Mon, 01 Jun 2020 18:13:11 +0000 Received: from hermes.gatewaynet.com ([193.92.121.98]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jfovs-0001F3-88 for pgsql-admin@lists.postgresql.org; Mon, 01 Jun 2020 18:13:10 +0000 Received: from hercules.internal.net (hercules.internal.net [10.9.200.8]) by hermes.gatewaynet.com (8.15.2/8.15.2/Debian-8) with ESMTP id 051ID5db014455 for ; Mon, 1 Jun 2020 21:13:05 +0300 Received: from [10.9.251.249] (dcmchgw.internal.net [10.9.0.1]) by hercules.internal.net (Postfix) with ESMTPS id 77AE61350B7E for ; Mon, 1 Jun 2020 21:13:04 +0300 (EEST) Subject: Re: pg_stat_database.xact_commit + pg_stat_database.xact_rollback as a reliable metric for xid consumption ? To: pgsql-admin@lists.postgresql.org References: <51fa222b-374e-39da-eb8c-66d667426bae@matrix.gatewaynet.com> <36dd7ae2-7c6f-86af-430b-dd9a0f7774b3@matrix.gatewaynet.com> From: Achilleas Mantzios Message-ID: <3d0096c7-f79f-5100-fe16-55677ce67006@matrix.gatewaynet.com> Date: Mon, 1 Jun 2020 21:13:02 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------C8704259D77F72E382A95F9D" Content-Language: en-US X-Virus-Scanned: clamav-milter 0.101.4 at hermes X-Virus-Status: Clean X-Spam-Status: No, score=-32.8 required=5.0 tests=ALL_TRUSTED,DYN_HAM_COORD, DYN_HAM_IT,DYN_HAM_IT_PARTNERS,DYN_HAM_NAUTICAL,DYN_SPAM_MED6, DYN_SPAM_MED7,DYN_SPAM_SUBJ,HTML_MESSAGE autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hermes.gatewaynet.com List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------C8704259D77F72E382A95F9D Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 1/6/20 5:52 μ.μ., Keith Fiske wrote: > > > On Mon, Jun 1, 2020 at 8:41 AM Achilleas Mantzios > > > wrote: > > Hello Julien > On 1/6/20 2:03 μ.μ., Julien Rouhaud wrote: > > On Mon, Jun 1, 2020 at 12:48 PM Achilleas Mantzios > > > wrote: > >> On 1/6/20 1:36 μ.μ., Julien Rouhaud wrote: > >>> On Mon, Jun 1, 2020 at 12:02 PM Achilleas Mantzios > >>> > wrote: > >>> Because read only transaction usually don't consume an xid.  > So yes > >>> pg_stat_database gives a more reasonable approximation of the real > >>> number of transactions happening on the server. > >> Thank you, so what would be the official way to monitor txid > consumption (from a MVCC / Freezing admin POV)? > > I'm not sure what you're really looking for. > I've been as explicit as it gets. When ppl talk about vacuum > freeze what they have in mind is xmin's and xmax's and the rate > that xids grow and consume the 2^31 address space, so I find it > strange > that such a basic metric is not included in the core statistics > collector. > That's all. > > > > > -- > Achilleas Mantzios > IT DEV Lead > IT DEPT > Dynacom Tankers Mgmt > > > > > I've done a writeup on effectively monitoring for xid > exhaustion/wraparound. > > https://info.crunchydata.com/blog/managing-transaction-id-wraparound-in-postgresql > Thank you, seems like a nice write-up, I'll check this out. > It is relatively simple to get the current xid state applying the > age() function to the pg_database catalog: > > SELECT datname, age(datfrozenxid), > current_setting('autovacuum_freeze_max_age') FROM pg_database ORDER BY > 2 DESC; > > But the query in the linked blog post provides things in a more > user-friendly manner for monitoring purposes as simple percentages > > -- > Keith Fiske > Senior Database Engineer > Crunchy Data - http://crunchydata.com --------------C8704259D77F72E382A95F9D Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit


On 1/6/20 5:52 μ.μ., Keith Fiske wrote:


On Mon, Jun 1, 2020 at 8:41 AM Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:
Hello Julien
On 1/6/20 2:03 μ.μ., Julien Rouhaud wrote:
> On Mon, Jun 1, 2020 at 12:48 PM Achilleas Mantzios
> <achill@matrix.gatewaynet.com> wrote:
>> On 1/6/20 1:36 μ.μ., Julien Rouhaud wrote:
>>> On Mon, Jun 1, 2020 at 12:02 PM Achilleas Mantzios
>>> <achill@matrix.gatewaynet.com> wrote:
>>> Because read only transaction usually don't consume an xid.  So yes
>>> pg_stat_database gives a more reasonable approximation of the real
>>> number of transactions happening on the server.
>> Thank you, so what would be the official way to monitor txid consumption (from a MVCC / Freezing admin POV)?
> I'm not sure what you're really looking for.
I've been as explicit as it gets. When ppl talk about vacuum freeze what they have in mind is xmin's and xmax's and the rate that xids grow and consume the 2^31 address space, so I find it strange
that such a basic metric is not included in the core statistics collector.
That's all.
>


--
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt




I've done a writeup on effectively monitoring for xid exhaustion/wraparound. 


Thank you, seems like a nice write-up, I'll check this out.
It is relatively simple to get the current xid state applying the age() function to the pg_database catalog:

SELECT datname, age(datfrozenxid), current_setting('autovacuum_freeze_max_age') FROM pg_database ORDER BY 2 DESC;

But the query in the linked blog post provides things in a more user-friendly manner for monitoring purposes as simple percentages

--
Keith Fiske
Senior Database Engineer
Crunchy Data - http://crunchydata.com
--------------C8704259D77F72E382A95F9D--