Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kakpy-0007gX-Ji for pgsql-hackers@arkaria.postgresql.org; Thu, 05 Nov 2020 19:22:22 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kakpx-00044U-Gp for pgsql-hackers@arkaria.postgresql.org; Thu, 05 Nov 2020 19:22:21 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kakpw-00044N-DU for pgsql-hackers@lists.postgresql.org; Thu, 05 Nov 2020 19:22:21 +0000 Received: from forward5-smtp.messagingengine.com ([66.111.4.239]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kakps-0005Ii-Sh for pgsql-hackers@lists.postgresql.org; Thu, 05 Nov 2020 19:22:19 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailforward.nyi.internal (Postfix) with ESMTP id DB2F619424F9; Thu, 5 Nov 2020 14:22:15 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 05 Nov 2020 14:22:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=g7eazdL3ULIoqCzPE3BXRozTo+wY3bUAzkDvlzSP0 74=; b=Vi3D0IUo4BLDfSNZii0aGEyprDET36AvFR9geO2/zWxKaOtpbL8GqSjek woyAuacL/Plsb3HEnZYpUT5C0qvrEe1oPhqtt37GKzPgrdvMhLMI5sPrladYtdtH xlp1shFN8Ii/vChf+Jhf0IVyjQDO7unq7rZ6KiQTt04Fjqoit7gGcOwNvl3zWz0C uuKmEUqmrFNbuzhOWJhjLKfZjEGjVGqUjR1W/xMPjWxEtpPZDniQXpPA3wja6O8r QDfRq7m4wkjHVglXvLc7wNjj13kHEsLWWC3E7dGjOd9Yqvbhb1kHzymNhSoW/6cp OIr7cbeF/fMtGXWKePkuVyW6CYpIg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedruddtjedguddvgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuvfhfhfhokffffgggjggtgfesthejredttdefjeenucfhrhhomheprfgv thgvrhcugfhishgvnhhtrhgruhhtuceophgvthgvrhdrvghishgvnhhtrhgruhhtsegvnh htvghrphhrihhsvggusgdrtghomheqnecuggftrfgrthhtvghrnhepheehieejgfevhfdv heelkeeuvefhheehvdehtefghfetfeeggefgveejjeffgfdvnecukfhppeekjedrudejje drjeehrdelgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhr ohhmpehpvghtvghrrdgvihhsvghnthhrrghuthesvghnthgvrhhprhhishgvuggsrdgtoh hm X-ME-Proxy: Received: from april.pezone.net (p57b14b5e.dip0.t-ipconnect.de [87.177.75.94]) by mail.messagingengine.com (Postfix) with ESMTPA id F33AC306005B; Thu, 5 Nov 2020 14:22:12 -0500 (EST) Subject: Re: libpq compression To: Andres Freund , Andrey Borodin References: <74F0A385-6ED4-4EA7-9498-1A8527F16472@yandex-team.ru> <22446ba0-058d-624a-a679-edc0215a56c8@postgrespro.ru> <20201028192712.xj2qneyb3fvmjm6c@alap3.anarazel.de> <3b1d2752-b7b5-db39-84c9-4ff024041fff@postgrespro.ru> <20201030210308.54jjmnvhspk6p7cb@alap3.anarazel.de> <059E5FF6-96A6-4DCE-83DE-8B20ECBFB843@yandex-team.ru> <20201102195059.w6qnplbyargqwbp2@alap3.anarazel.de> Cc: Konstantin Knizhnik , PostgreSQL Hackers , =?UTF-8?B?0JTQsNC90LjQuNC7INCX0LDRhdC70YvRgdGC0L7Qsg==?= From: Peter Eisentraut Organization: EnterpriseDB Message-ID: <5c832f37-db91-edec-3b31-7caef365df69@enterprisedb.com> Date: Thu, 5 Nov 2020 20:22:10 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20201102195059.w6qnplbyargqwbp2@alap3.anarazel.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 2020-11-02 20:50, Andres Freund wrote: > On 2020-10-31 22:25:36 +0500, Andrey Borodin wrote: >> But the price of compression is 1 cpu for 500MB/s (zstd). With a >> 20Gbps network adapters cost of recompressing all traffic is at most >> ~4 cores. > > It's not quite that simple, because presumably each connection is going > to be handled by one core at a time in the pooler. So it's easy to slow > down peak throughput if you also have to deal with TLS etc. Also, current deployments of connection poolers use rather small machine sizes. Telling users you need 4 more cores per instance now to decompress and recompress all the traffic doesn't seem very attractive. Also, it's not unheard of to have more than one layer of connection pooling. With that, this whole design sounds a bit like a heat-generation system. ;-)