pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Daniil Zakhlystov <usernamedt@yandex-team.ru>
Cc: Andres Freund <andres@anarazel.de>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Andrey M. Borodin <x4mmm@yandex-team.ru>
Cc: o.bartunov@postgrespro.ru <o.bartunov@postgrespro.ru>
Subject: Re: libpq compression
Date: Thu, 5 Nov 2020 19:01:43 +0300
Message-ID: <a604ecba-86a0-d9e8-4fab-6cb4fa77cbc1@postgrespro.ru> (raw)
In-Reply-To: <CAEze2WgFEOVwQZap+ZQFuA7A_uLDSAQRSNm3EqCVgrpeM=TGNQ@mail.gmail.com>
References: <c0230242-2279-d6f7-e1ff-d3de908bdfbc@postgrespro.ru>
	<a112e0a2-6908-5a0d-ac22-012ebb6169b0@postgrespro.ru>
	<74F0A385-6ED4-4EA7-9498-1A8527F16472@yandex-team.ru>
	<a13ce4d1-768c-1449-25f6-0cd645499d93@postgrespro.ru>
	<fa9113c0-db7a-2ed8-111c-a082a2217e4b@postgrespro.ru>
	<22446ba0-058d-624a-a679-edc0215a56c8@postgrespro.ru>
	<20201028192712.xj2qneyb3fvmjm6c@alap3.anarazel.de>
	<3b1d2752-b7b5-db39-84c9-4ff024041fff@postgrespro.ru>
	<20201030210308.54jjmnvhspk6p7cb@alap3.anarazel.de>
	<0ae2162b-818f-6a6d-cca6-ee4e73733e8e@postgrespro.ru>
	<67498D86-006D-4D56-85E2-88ABA07B2715@yandex-team.ru>
	<34328cc0-9669-f30d-50f6-46e9f3f21cde@postgrespro.ru>
	<ED4564BD-0DA3-4630-8AA3-9051D29717F7@yandex-team.ru>
	<9d33e21e-54b2-fcf3-b90f-ced286df00c5@postgrespro.ru>
	<CAEze2WiavC64KuNGKrW-M1DPb=7GiF9XshbF5CDS2ADrMJQvSQ@mail.gmail.com>
	<bc017bd0-c00c-44a8-0a81-69319135446d@postgrespro.ru>
	<CAEze2WgFEOVwQZap+ZQFuA7A_uLDSAQRSNm3EqCVgrpeM=TGNQ@mail.gmail.com>



On 05.11.2020 15:40, Matthias van de Meent wrote:
>
> How about Xx_logical_bytes for raw the pg command stream data, and
> keeping Xx_compressed_bytes for the compressed data in/out?

Frankly speaking I do not like work "logical" in this context.
It is in any case physical bytes, received from the peer.
Speaking about compression or encryption, "raw" is much widely used
for uncompressed or plain data.

>> This view pg_stat_network_traffic reports traffic from server (backend)
>> point of view, i.e.
>> rx_bytes (received bytes) are commands sent from client to the server
>> tx_bytes (transmitted bytes) are responses sent by server to the client.
>>
>> If compression is not used then rx_compressed_bytes =
>> tx_compressed_bytes = 0
>> It seems to be more natural then assigning them the same values as (raw
>> bytes).
>> Because it can really happen that for BLOBs with already compressed data
>> (video images or sound)
>> compressed data will be almost the same as raw data even if compression
>> is enabled.
>> So it seems to be important to distinguished situations when data can
>> not be compressed and
>> when it is not compressed at all.
> Looking at it from that viewpoint, I agree. My primary reason for
> suggesting this was that it would be useful to expose how much data
> was transferred between the client and the server, which cannot be
> constructed from that view for compression-enabled connections. That
> is because the compression methods' counting only starts after some
> bytes have already been transferred, and the raw/logical counter
> starts deviating once compression is enabled.

Sorry, I do not understand your point.
This view reports network traffic from server's side.
But client's traffic information is "mirror" of this statistic: 
server_tx=client_rx and visa versa.

Yes, first few bytes exchanged by client and server during handshake are 
not compressed.
But them are correctly calculated as "raw bytes". And certainly this few 
bytes can not have any influence on
measured average compression ratio (the main goal of using this network 
traffic statistic from my point of view).


-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

view thread (289+ messages)  latest in thread

Message-ID: <a604ecba-86a0-d9e8-4fab-6cb4fa77cbc1@postgrespro.ru>
Permalink:  ../a604ecba-86a0-d9e8-4fab-6cb4fa77cbc1@postgrespro.ru/
Also on:    postgresql.org/message-id/a604ecba-86a0-d9e8-4fab-6cb4fa77cbc1@postgrespro.ru

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: k.knizhnik@postgrespro.ru, boekewurm+postgres@gmail.com, usernamedt@yandex-team.ru, andres@anarazel.de, pgsql-hackers@lists.postgresql.org, x4mmm@yandex-team.ru, o.bartunov@postgrespro.ru
  Subject: Re: libpq compression
  In-Reply-To: <a604ecba-86a0-d9e8-4fab-6cb4fa77cbc1@postgrespro.ru>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox