Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hEa8A-0005zT-Ja for pgsql-hackers@arkaria.postgresql.org; Thu, 11 Apr 2019 13:52:43 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1hEa89-0005Vb-5q for pgsql-hackers@arkaria.postgresql.org; Thu, 11 Apr 2019 13:52:41 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hEa88-0005VU-KH for pgsql-hackers@lists.postgresql.org; Thu, 11 Apr 2019 13:52:40 +0000 Received: from cyclops.postgrespro.ru ([93.174.131.138] helo=mail.postgrespro.ru) by makus.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1hEa84-0002VJ-S0 for pgsql-hackers@postgresql.org; Thu, 11 Apr 2019 13:52:39 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id 3B51F21C3FAD; Thu, 11 Apr 2019 16:52:34 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-99 required=4 WHITELISTED tests=[] autolearn=unavailable Received: from [192.168.27.200] (gw.postgrespro.ru [93.174.131.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id 0C71F21C3FA3; Thu, 11 Apr 2019 16:52:34 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1554990754; bh=6WW8J/YKZs7hjp0k8zngh+/RD9OMJslWAH2YXaN1iPE=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=OzsYs3M5qF0ch5XkAFtIZeinN04u4+j1lF6GdjMcA+/N7HL6/QEKXZoVDm4uM0AMt wOFUo73XMaErvUrim3fYznUREHeAW/51PCaZFIMAv0XHhz9JexSJ7EoH9tDjYczRci Y+VmEh9XCEJ5tECAgOO7J3ZjrrXsj97y1XOLv1FU= Subject: Re: Zedstore - compressed in-core columnar storage To: Andreas Karlsson , Ashwin Agrawal , Mark Kirkwood Cc: PostgreSQL mailing lists References: <6A2B85A8-1ABD-4F85-981E-C94172D662FD@pivotal.io> <44cafbdf-0a04-64ef-3cf8-6d5b14b68643@postgrespro.ru> <93222d26-2141-acf2-61ef-3127f57e80e6@proxel.se> From: Konstantin Knizhnik Message-ID: <59ef6173-6667-8a02-ff7c-a52c8ce05a4f@postgrespro.ru> Date: Thu, 11 Apr 2019 16:52:33 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <93222d26-2141-acf2-61ef-3127f57e80e6@proxel.se> Content-Type: multipart/alternative; boundary="------------BC5B816407BAFA7F897DE3AE" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------BC5B816407BAFA7F897DE3AE Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 11.04.2019 16:18, Andreas Karlsson wrote: > > On 4/11/19 10:46 AM, Konstantin Knizhnik wrote: > >> This my results of compressing pbench data using different compressors: >> >> Configuration Size (Gb) Time (sec) >> no compression >> 15.31 92 >> zlib (default level) 2.37 284 >> zlib (best speed) 2.43 191 >> postgres internal lz 3.89 214 >> lz4 4.12 >> 95 >> snappy 5.18 99 >> lzfse 2.80 1099 >> (apple) 2.80 1099 >> 1.69 125 >> >> >> >> You see that zstd provides almost 2 times better compression ration >> and almost at the same speed. > > > What is "(apple) 2.80 1099"? Was that intended to be zstd? > > Andreas > Ugh... Cut and paste problems. The whole document can be found here: http://garret.ru/PageLevelCompression.pdf lzfse (apple)       2.80    1099 zstd (facebook)  1.69    125 ztsd is compression algorithm proposed by facebook: https://github.com/facebook/zstd Looks like it provides the best speed/compress ratio result. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company --------------BC5B816407BAFA7F897DE3AE Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

On 11.04.2019 16:18, Andreas Karlsson wrote:

On 4/11/19 10:46 AM, Konstantin Knizhnik wrote:

This my results of compressing pbench data using different compressors:

Configuration Size (Gb) Time (sec)
no compression
 15.31 92
zlib (default level) 2.37 284
zlib (best speed) 2.43 191
postgres internal lz 3.89 214
lz4 4.12
95
snappy 5.18 99
lzfse 2.80 1099
(apple) 2.80 1099
 1.69 125


You see that zstd provides almost 2 times better compression ration and almost at the same speed.


What is "(apple) 2.80 1099"? Was that intended to be zstd?

Andreas

Ugh...
Cut and paste problems.
The whole document can be found here: http://garret.ru/PageLevelCompression.pdf

lzfse (apple)       2.80    1099
zstd (facebook)  1.69    125

ztsd is compression algorithm proposed by facebook:  https://github.com/facebook/zstd
Looks like it provides the best speed/compress ratio result.
-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
--------------BC5B816407BAFA7F897DE3AE--