Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kahhx-00012z-KY for pgsql-hackers@arkaria.postgresql.org; Thu, 05 Nov 2020 16:01:54 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kahhw-00035F-BC for pgsql-hackers@arkaria.postgresql.org; Thu, 05 Nov 2020 16:01:52 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kahhw-000357-1U for pgsql-hackers@lists.postgresql.org; Thu, 05 Nov 2020 16:01:52 +0000 Received: from mail.postgrespro.ru ([93.174.131.139]) by magus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kahhp-0007Np-45 for pgsql-hackers@lists.postgresql.org; Thu, 05 Nov 2020 16:01:51 +0000 Received: from [192.168.27.200] (unknown [93.174.131.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id 0BD3521C49EF; Thu, 5 Nov 2020 19:01:44 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1604592104; bh=RGK9FSShRTqim8F3wy8WXTNi46HthFeN7DKAjWtaQFw=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=D3MWaKa2vvvC7s9yShSSuqiL3gV+ufavtPVZ1HZEiQKnym5Gsim+Z6yM+YOp9/sbY OcAF5EQkuWPgkh6JohWJ3SGepvKlnsmDzQLjikyeuIH6V7KjLCE+6TadPp8LRvjiJa kgLJBvT3aRJCYOZlIIlGR4iaIv6XG3h1tdHRTL6Q= Subject: Re: libpq compression To: Matthias van de Meent Cc: Daniil Zakhlystov , Andres Freund , PostgreSQL Hackers , "Andrey M. Borodin" , "o.bartunov@postgrespro.ru" References: <74F0A385-6ED4-4EA7-9498-1A8527F16472@yandex-team.ru> <22446ba0-058d-624a-a679-edc0215a56c8@postgrespro.ru> <20201028192712.xj2qneyb3fvmjm6c@alap3.anarazel.de> <3b1d2752-b7b5-db39-84c9-4ff024041fff@postgrespro.ru> <20201030210308.54jjmnvhspk6p7cb@alap3.anarazel.de> <0ae2162b-818f-6a6d-cca6-ee4e73733e8e@postgrespro.ru> <67498D86-006D-4D56-85E2-88ABA07B2715@yandex-team.ru> <34328cc0-9669-f30d-50f6-46e9f3f21cde@postgrespro.ru> <9d33e21e-54b2-fcf3-b90f-ced286df00c5@postgrespro.ru> From: Konstantin Knizhnik Message-ID: Date: Thu, 5 Nov 2020 19:01:43 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------12677AAD6B73982881677DF5" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------12677AAD6B73982881677DF5 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 05.11.2020 15:40, Matthias van de Meent wrote: > > How about Xx_logical_bytes for raw the pg command stream data, and > keeping Xx_compressed_bytes for the compressed data in/out? Frankly speaking I do not like work "logical" in this context. It is in any case physical bytes, received from the peer. Speaking about compression or encryption, "raw" is much widely used for uncompressed or plain data. >> This view pg_stat_network_traffic reports traffic from server (backend) >> point of view, i.e. >> rx_bytes (received bytes) are commands sent from client to the server >> tx_bytes (transmitted bytes) are responses sent by server to the client. >> >> If compression is not used then rx_compressed_bytes = >> tx_compressed_bytes = 0 >> It seems to be more natural then assigning them the same values as (raw >> bytes). >> Because it can really happen that for BLOBs with already compressed data >> (video images or sound) >> compressed data will be almost the same as raw data even if compression >> is enabled. >> So it seems to be important to distinguished situations when data can >> not be compressed and >> when it is not compressed at all. > Looking at it from that viewpoint, I agree. My primary reason for > suggesting this was that it would be useful to expose how much data > was transferred between the client and the server, which cannot be > constructed from that view for compression-enabled connections. That > is because the compression methods' counting only starts after some > bytes have already been transferred, and the raw/logical counter > starts deviating once compression is enabled. Sorry, I do not understand your point. This view reports network traffic from server's side. But client's traffic information is "mirror" of this statistic: server_tx=client_rx and visa versa. Yes, first few bytes exchanged by client and server during handshake are not compressed. But them are correctly calculated as "raw bytes". And certainly this few bytes can not have any influence on measured average compression ratio (the main goal of using this network traffic statistic from my point of view). -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company --------------12677AAD6B73982881677DF5 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

On 05.11.2020 15:40, Matthias van de Meent wrote:

How about Xx_logical_bytes for raw the pg command stream data, and
keeping Xx_compressed_bytes for the compressed data in/out?

Frankly speaking I do not like work "logical" in this context.
It is in any case physical bytes, received from the peer.
Speaking about compression or encryption, "raw" is much widely used
for uncompressed or plain data. 


      

        
This view pg_stat_network_traffic reports traffic from server (backend)
point of view, i.e.
rx_bytes (received bytes) are commands sent from client to the server
tx_bytes (transmitted bytes) are responses sent by server to the client.

If compression is not used then rx_compressed_bytes =
tx_compressed_bytes = 0
It seems to be more natural then assigning them the same values as (raw
bytes).
Because it can really happen that for BLOBs with already compressed data
(video images or sound)
compressed data will be almost the same as raw data even if compression
is enabled.
So it seems to be important to distinguished situations when data can
not be compressed and
when it is not compressed at all.


      

      
Looking at it from that viewpoint, I agree. My primary reason for
suggesting this was that it would be useful to expose how much data
was transferred between the client and the server, which cannot be
constructed from that view for compression-enabled connections. That
is because the compression methods' counting only starts after some
bytes have already been transferred, and the raw/logical counter
starts deviating once compression is enabled.


    

    

    Sorry, I do not understand your point.

    This view reports network traffic from server's side.

    But client's traffic information is "mirror" of this statistic:
    server_tx=client_rx and visa versa.

    

    Yes, first few bytes exchanged by client and server during handshake
    are not compressed.

    But them are correctly calculated as "raw bytes". And certainly this
    few bytes can not have any influence on 

    measured average compression ratio (the main goal of using this
    network traffic statistic from my point of view).

    




    

    
-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company 

  


--------------12677AAD6B73982881677DF5--