Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fWQIq-00079s-AC for pgsql-hackers@arkaria.postgresql.org; Fri, 22 Jun 2018 17:56:56 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fWQIn-0000i5-UL for pgsql-hackers@arkaria.postgresql.org; Fri, 22 Jun 2018 17:56:53 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fWQIj-0000IG-Ew for pgsql-hackers@lists.postgresql.org; Fri, 22 Jun 2018 17:56:49 +0000 Received: from mx1.redhat.com ([209.132.183.28]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fWQId-0005od-Rf for pgsql-hackers@lists.postgresql.org; Fri, 22 Jun 2018 17:56:47 +0000 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EA49B3082125; Fri, 22 Jun 2018 17:56:40 +0000 (UTC) Received: from localhost (ovpn-112-20.rdu2.redhat.com [10.10.112.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F12D7B8E6; Fri, 22 Jun 2018 17:56:40 +0000 (UTC) From: Robbie Harwood To: Konstantin Knizhnik , Thomas Munro Cc: Grigory Smolkin , PostgreSQL Hackers Subject: Re: libpq compression In-Reply-To: <852c6ee2-5971-dc56-923b-f8eeea6b8961@postgrespro.ru> References: <429cf55b-a8bf-c4ba-196d-5b45420145a1@postgrespro.ru> <36db5d02-45e6-b863-1418-825ef20ae4c6@postgrespro.ru> <3b0260da-fe47-2341-da01-51383fe9172d@postgrespro.ru> <15419613-35b5-0ff6-6261-834abf73330c@postgrespro.ru> <9b3a735c-06c2-631f-8e97-e3982e575210@postgrespro.ru> <5b673a96-b16e-9a79-c0ec-f7f03f207c62@postgrespro.ru> <852c6ee2-5971-dc56-923b-f8eeea6b8961@postgrespro.ru> Date: Fri, 22 Jun 2018 13:56:37 -0400 Message-ID: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Fri, 22 Jun 2018 17:56:41 +0000 (UTC) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --=-=-= Content-Type: text/plain Konstantin Knizhnik writes: > On 22.06.2018 18:59, Robbie Harwood wrote: >> Konstantin Knizhnik writes: >>> On 21.06.2018 20:14, Robbie Harwood wrote: >>>> Konstantin Knizhnik writes: >>>>> On 21.06.2018 17:56, Robbie Harwood wrote: >>>>>> Konstantin Knizhnik writes: >>>>>>> On 20.06.2018 23:34, Robbie Harwood wrote: >>>>>>>> Konstantin Knizhnik writes: >>>>>>>> >>>>>>>>> My idea was the following: client want to use compression. But >>>>>>>>> server may reject this attempt (for any reasons: it doesn't >>>>>>>>> support it, has no proper compression library, do not want to >>>>>>>>> spend CPU for decompression,...) Right now compression >>>>>>>>> algorithm is hardcoded. But in future client and server may >>>>>>>>> negotiate to choose proper compression protocol. This is why >>>>>>>>> I prefer to perform negotiation between client and server to >>>>>>>>> enable compression. >>>>>>>> >>>>>>>> Well, for negotiation you could put the name of the algorithm >>>>>>>> you want in the startup. It doesn't have to be a boolean for >>>>>>>> compression, and then you don't need an additional round-trip. >>>>>>> >>>>>>> Sorry, I can only repeat arguments I already mentioned: >>>>>>> >>>>>>> - in future it may be possible to specify compression algorithm >>>>>>> >>>>>>> - even with boolean compression option server may have some >>>>>>> reasons to reject client's request to use compression >>>>>>> >>>>>>> Extra flexibility is always good thing if it doesn't cost too >>>>>>> much. And extra round of negotiation in case of enabling >>>>>>> compression seems to me not to be a high price for it. >>>>>> >>>>>> You already have this flexibility even without negotiation. I >>>>>> don't want you to lose your flexibility. Protocol looks like >>>>>> this: >>>>>> >>>>>> - Client sends connection option "compression" with list of >>>>>> algorithms it wants to use (comma-separated, or something). >>>>>> >>>>>> - First packet that the server can compress one of those algorithms >>>>>> (or none, if it doesn't want to turn on compression). >>>>>> >>>>>> No additional round-trips needed. >>>>> >>>>> This is exactly how it works now... Client includes compression >>>>> option in connection string and server replies with special >>>>> message ('Z') if it accepts request to compress traffic between >>>>> this client and server. >>>> >>>> No, it's not. You don't need this message. If the server receives >>>> a compression request, it should just turn compression on (or not), >>>> and then have the client figure out whether it got compression >>>> back. >>> >>> How it will managed to do it. It receives some reply and first of >>> all it should know whether it has to be decompressed or not. >> >> You can tell whether a message is compressed by looking at it. The >> way the protocol works, every message has a type associated with it: >> a single byte, like 'R', that says what kind of message it is. > > Compressed message can contain any sequence of bytes, including 'R':) Then tag your messages with a type byte. Or do it the other way around - look for the zstd framing within a message. Please, try to work with me on this instead of fighting every design change. Thanks, --Robbie --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIyBAEBCgAdFiEEA5qc6hnelQjDaHWqJTL5F2qVpEIFAlstOFUACgkQJTL5F2qV pELM4w/4g87q0WCcYZPC1srM1+GV9jxVRNM7+14haL0H+XLKAMPVlBIjeAjvpHJ/ gmljPAceTx4z6v8jRgH6tN6INIaiuNuObOlpLzvfVX1rupNmMz4+Ev37no3KmgxN 1iqiTO4PdZ06dABlcEMghrRZOxWEjZIU8rfalwUatExa5XiKazIqx+gLL+YwxVI+ EDWR8Myx1DCWA2e9zsx4EwhQ1jUZuyE81Wsqqz7ieTEcSb7QWK7FYeYWBgN4Vbo5 j/NI5YbR3ZJl2aOw8rAZ7BvHX1wc34bqop+BootrfSNrWSpZVoym9WIrkEm5r/yW 5aYHcAMpv1TKErRNwuYx3S1hcE8mF9jwhzRXvgdlws2mHxv0qrrVIJJSNS4lpF6A HuRQvwm2/JiNCjyU0OFqxOTzn0gK/FQ5Yhkxmz+uSMRbAAlmwLXeRllm/Jp2gMAI MkpmBt4mYoaZjUnWTBrFI+caVxAndpmypcPAnavi4PAlu3G0X3qmflU+1wo20L51 q9PvaeUIuN06oEhGAFAJSzYTWxGhbbMHchJ0ua5BUu538Qe9hAdXkCcOY+bQ6tHy CNR8Q8xAoR2d95l1c2dynq+grCjb8pfZ794NgSF6c4fHSSydF6S3xgTs1oABKCyV CC9evMRq+3lUs5siD9pmyE1OhdaA5cThTO2HySFRI430vS/xnA== =38wV -----END PGP SIGNATURE----- --=-=-=--