Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gs96K-0007Rn-2T for pgsql-hackers@arkaria.postgresql.org; Fri, 08 Feb 2019 16:34:05 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gs96I-0008Jd-TD for pgsql-hackers@arkaria.postgresql.org; Fri, 08 Feb 2019 16:34:02 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gs96I-0008HZ-IX for pgsql-hackers@lists.postgresql.org; Fri, 08 Feb 2019 16:34:02 +0000 Received: from mail.postgrespro.ru ([93.174.131.138]) by magus.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gs96B-0004RN-MI for pgsql-hackers@lists.postgresql.org; Fri, 08 Feb 2019 16:34:02 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id 65D6F21CB651; Fri, 8 Feb 2019 19:33:54 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-99 required=4 WHITELISTED tests=[] autolearn=unavailable Received: from [192.168.27.200] (gw.postgrespro.ru [93.174.131.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id 79A7F21CB611; Fri, 8 Feb 2019 19:33:53 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1549643634; bh=oqGpjC7ov5K6ISKyGXeUVAw13vxd7dTr3e9VGnfrBMQ=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=Lfme7AA8BSdu5p/cjwuvrfIHd0cYO8bsv5EoUjCh7nMok7CxdgIEyjV9CL+gyBDeQ CqaCMQxSWiDZXbFsIJE30se5HKCOwEd6TXcTbkaelmD8TAJxlSzidFf5BbCjzxoamn 9wrrHxxGf4djHLmZLquHxv5mZDDGr38WUMrE00Io= Subject: Re: libpq compression To: Daniel Gustafsson Cc: "Iwata, Aya" , Michael Paquier , Andrew Dunstan , Robert Haas , "rharwood@redhat.com" , Thomas Munro , "g.smolkin@postgrespro.ru" , PostgreSQL Developers , Dmitry Dolgov <9erthalion6@gmail.com> References: <1eca5819-2b58-e929-1bf3-2d4dadf491b9@2ndQuadrant.com> <8d9386be-a567-c526-2fc1-153d6ffd3507@2ndQuadrant.com> <17747c8b-c088-6f92-38e9-98316910202a@postgrespro.ru> <20181001064903.GG11712@paquier.xyz> <71E660EB361DF14299875B198D4CE5423DEB752E@g01jpexmbkw25> <1b2d418b-f3b1-620e-8456-145d6e943da6@postgrespro.ru> <71E660EB361DF14299875B198D4CE5423DED67E5@g01jpexmbkw25> From: Konstantin Knizhnik Message-ID: Date: Fri, 8 Feb 2019 19:33:53 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 08.02.2019 12:33, Daniel Gustafsson wrote: >> On 8 Feb 2019, at 10:15, Konstantin Knizhnik wrote: >> Frankly speaking, I do not think that such flexibility in choosing compression algorithms is really needed. >> I do not expect that there will be many situations where old client has to communicate with new server or visa versa. >> In most cases both client and server belongs to the same postgres distributive and so implements the same compression algorithm. >> As far as we are compressing only temporary data (traffic), the problem of providing backward compatibility seems to be not so important. > I don’t think this assumption is entirely valid, and would risk unnecessary > breakage. I am also not sure in this assumption. We (PostgresPro) are having some issues now with CFS (file level compression of Postgres database). Some build are using zstd, some are using zlib (default)... zstd is faster and provides better compression ratio and is available at most platforms. But zlib is available almost everywhere and is used by Postgres by default... The only thing I know for sure: if we implement several algorithms and make it possible for database user to make a choice, then we will get much more problems. Right now Postgres is using zlib as the only supported compression algorithm in many places. So may be libpq compression should also use only zlib and provide no other choices? Concerning backward compatibility. Assume that we allow use zstd, but then Facebook change zstd license or some critical bug in it is found. So we will have to exclude dependency on zstd. So there will be no backward compatibility in any case, even if we support more sophisticated negotiation between client and server in choosing compression algorithm. What can be more interesting - is to support custom compression algorithms (optimized for the particular data flow). But it seems to be completely different and much more sophisticated story. We have to provide some mechanism for loading foreign libraries at client side! IMHO it is overkill. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company