From: Tom Lane <tgl@sss.pgh.pa.us>
To: Ashwin Agrawal <aagrawal@pivotal.io>
cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>,
        PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Subject: Re: Zedstore - compressed in-core columnar storage
In-reply-to: <6A2B85A8-1ABD-4F85-981E-C94172D662FD@pivotal.io>
References: 
 <CALfoeiuF-m5jg51mJUPm5GN8u396o5sA2AF5N97vTRAEDYac7w@mail.gmail.com>
 <fc2d4f99-1ca6-854f-7025-ec3db94f6e04@catalyst.net.nz>
 <f301066e-08e1-a4b6-e058-0c1f24a1cc92@catalyst.net.nz>
 <6A2B85A8-1ABD-4F85-981E-C94172D662FD@pivotal.io>
Comments: In-reply-to Ashwin Agrawal <aagrawal@pivotal.io>
	message dated "Wed, 10 Apr 2019 22:03:26 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-ID: <11256.1554994460.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Thu, 11 Apr 2019 10:54:20 -0400
Message-ID: <11257.1554994460@sss.pgh.pa.us>
Precedence: bulk

Ashwin Agrawal <aagrawal@pivotal.io> writes:
> Thank you for trying it out. Yes, noticed for certain patterns pg_lzcomp=
ress() actually requires much larger output buffers. Like for one 86 len s=
ource it required 2296 len output buffer. Current zedstore code doesn=E2=80=
=99t handle this case and errors out. LZ4 for same patterns works fine, wo=
uld highly recommend using LZ4 only, as anyways speed is very fast as well=
 with it.

You realize of course that *every* compression method has some inputs that
it makes bigger.  If your code assumes that compression always produces a
smaller string, that's a bug in your code, not the compression algorithm.

			regards, tom lane