From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
To: Robbie Harwood <rharwood@redhat.com>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Grigory Smolkin <g.smolkin@postgrespro.ru>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: libpq compression
Date: Fri, 15 Feb 2019 17:03:24 +0300
Message-ID: <cd974e7e-85da-19a4-f25b-a7a56ff87d00@postgrespro.ru> (raw)
In-Reply-To: <ebc8ebd2-1a3b-43f7-d1a1-6e8b9ed65e62@2ndquadrant.com>
References: <aad16e41-b3f9-e89d-fa57-fb4c694bec25@postgrespro.ru>
<dbf9035f-7f7b-b053-cc2a-52e5147d4557@postgrespro.ru>
<429cf55b-a8bf-c4ba-196d-5b45420145a1@postgrespro.ru>
<CAEepm=3CGUX4Vyw=jFaOedt99HS1HQiBKYTX4OYHo7HU2AczZg@mail.gmail.com>
<36db5d02-45e6-b863-1418-825ef20ae4c6@postgrespro.ru>
<CAEepm=3RAS8xVQK8rs+Xitzp+JXzGdGsMA7f+H8fjKQ7Do8aSA@mail.gmail.com>
<f4ad0809-dd90-e321-5c1a-0ca68a136a8f@postgrespro.ru>
<jlg4lhzdeak.fsf@redhat.com>
<3b0260da-fe47-2341-da01-51383fe9172d@postgrespro.ru>
<ebc8ebd2-1a3b-43f7-d1a1-6e8b9ed65e62@2ndquadrant.com>
On 15.02.2019 15:42, Peter Eisentraut wrote:
> On 2018-06-19 09:54, Konstantin Knizhnik wrote:
>> The main drawback of streaming compression is that you can not
>> decompress some particular message without decompression of all previous
>> messages.
> It seems this would have an adverse effect on protocol-aware connection
> proxies: They would have to uncompress everything coming in and
> recompress everything going out.
>
> The alternative of compressing each packet individually would work much
> better: A connection proxy could peek into the packet header and only
> uncompress the (few, small) packets that it needs for state and routing.
>
Individual compression of each message depreciate all idea of libpq
compression.
Messages are two small to efficiently compress each of them separately.
So using streaming compression algorithm is absolutely necessary here.
Concerning possible problem with proxies I do not think that it is
really a problem.
Proxy is very rarely located somewhere in the "middle" between client
and database servers.
It is usually launched either in the same network as DBMS client (for
example, if client is application server) either in the same network
with database server.
In both cases there is not so much sense to pass compressed traffic
through the proxy.
If proxy and DBMS server are located in the same network, then proxy
should perform decompression and send
decompressed messages to the database server.
Thank you very much for noticing this problem with compatibility
compression and protocol-aware connection proxies.
I have wrote that current compression implementation (zpq_stream.c) can
be used not only for libpq backend/frontend, but
also for compression any other streaming data. But I could not imaging
what other data sources can require compression.
And proxy is exactly such case: it also needs to compress/decompress
messages.
It is one more argument to make interface of zpq_stream as simple as
possible and encapsulate all inflating/deflating logic in this code.
It can be achieved by passing arbitrary rx/tx function to zpq_create
function.
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: k.knizhnik@postgrespro.ru, peter.eisentraut@2ndquadrant.com, rharwood@redhat.com, thomas.munro@enterprisedb.com, g.smolkin@postgrespro.ru, pgsql-hackers@lists.postgresql.org
Subject: Re: libpq compression
In-Reply-To: <cd974e7e-85da-19a4-f25b-a7a56ff87d00@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox