Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1krnQ6-0002oP-QI for pgsql-hackers@arkaria.postgresql.org; Tue, 22 Dec 2020 19:34:06 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1krnPZ-0004yf-Dd for pgsql-hackers@arkaria.postgresql.org; Tue, 22 Dec 2020 19:33:33 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1krnPZ-0004yY-70 for pgsql-hackers@lists.postgresql.org; Tue, 22 Dec 2020 19:33:33 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1krnPU-0006lT-J4 for pgsql-hackers@lists.postgresql.org; Tue, 22 Dec 2020 19:33:32 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 0BMJXMlD630702; Tue, 22 Dec 2020 14:33:23 -0500 From: Tom Lane To: Tomas Vondra cc: Andrey Borodin , Robert Haas , Daniil Zakhlystov , Konstantin Knizhnik , Denis Smirnov , PostgreSQL Hackers Subject: Re: libpq compression In-reply-to: <1f76b4b8-3bff-17af-ef63-872190411a60@enterprisedb.com> References: <612f227d-cb27-9238-5922-1500f8b2eb96@postgrespro.ru> <160623927937.7563.15974279758478581796.pgcf@coridan.postgresql.org> <6811D196-E2FB-40CC-B1C9-F19427FBF675@yandex-team.ru> <93027ee4-fefa-a8ca-2cc7-245c912bf824@postgrespro.ru> <62527092-16BD-479F-B503-FA527AF3B0C2@yandex-team.ru> <579f308e-c46c-2e79-3f90-9846450a3d71@enterprisedb.com> <626075.1608663785@sss.pgh.pa.us> <1! f76b4b8-3bff-17af-ef63-872190411a60@enterprisedb.com> Comments: In-reply-to Tomas Vondra message dated "Tue, 22 Dec 2020 20:25:58 +0100" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <630700.1608665602.1@sss.pgh.pa.us> Date: Tue, 22 Dec 2020 14:33:22 -0500 Message-ID: <630701.1608665602@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk Tomas Vondra writes: > On 12/22/20 8:03 PM, Tom Lane wrote: >> The link Ken pointed at suggests that restricting the window size to >> 8MB is a common compromise. It's not clear to me what that does to >> the achievable compression ratio. Even 8MB could be an annoying cost >> if it's being paid per-process, on both the server and client sides. > Possibly, but my understanding is that's merely a recommendation for the > decoder library (e.g. libzstd), and it's not clear to me if/how that > relates to the compression level or how to influence it. > From the results shared by Daniil, the per-client overhead seems way > higher than 8MB, so either libzstd does not respect this recommendation > or maybe there's something else going on. I'd assume that there's a direct correlation between the compression level setting and the window size; but I've not studied the libzstd docs in enough detail to know what it is. regards, tom lane