public inbox for [email protected]
help / color / mirror / Atom feedFrom: David Steele <[email protected]>
To: Peter Geoghegan <[email protected]>
To: Robert Haas <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: adding 'zstd' as a compression algorithm
Date: Tue, 15 Feb 2022 18:09:07 -0600
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAH2-WzmsBX_7fdSMmuPDAZg9v+4bfbdgBOVeM4LJcx2KmbtPQg@mail.gmail.com>
References: <CA+TgmoatQKGd+8SjcV+bzvw4XaoEwminHjU83yG12+NXtQzTTQ@mail.gmail.com>
<CAH2-Wz=B3-Q9hK6q4VWYu66c4ra=AUzDRUw4H8o1ESz8Dbk8Yw@mail.gmail.com>
<CA+TgmoYo1unP9y_tDATd7G7SOycxtzbtT0vsc+ZZ1G6pftXRBw@mail.gmail.com>
<CAH2-WzmsBX_7fdSMmuPDAZg9v+4bfbdgBOVeM4LJcx2KmbtPQg@mail.gmail.com>
On 2/15/22 16:10, Peter Geoghegan wrote:
> On Tue, Feb 15, 2022 at 12:00 PM Robert Haas <[email protected]> wrote:
>> I'm not sure I completely follow this. There are cases where we use
>> compression algorithms for internal purposes, and we can change the
>> defaults without users knowing or caring. For example, we could decide
>> that LZ4-compressing FPIs in WAL records is a sensible default and
>> just start doing it, and users need not care.
I'm not sure that this is entirely true. lz4 is available on most
non-EOL distros but you don't need to look to far to find a distro that
does not have it. So, choosing lz4 by default could impact binary
portability. Of course, there are lots of other factors that impact
binary portability, e.g. collations, but this would add one more.
This is even more true for zstd since it is not as ubiquitous as lz4. In
fact, it is not even included with base RHEL8. You need to install EPEL
to get zstd.
>> But backups are
>> different, because when you pg_basebackup -Ft, you get .tar or .tar.gz
>> or .tar.lz4 files which we don't give you tools to extract.
>
> What I meant is that you're buying into an ecosystem by choosing to
> use a tool like pgBackrest. That might not be the right thing to do
> universally, but it comes pretty close. You as a user are largely
> deferring to the maintainers and their choice of defaults. Not
> entirely, of course, but to a significant degree, especially in
> matters like this. There aren't that many dimensions to the problem
> once the question of compatibility is settled (it's settled here
> because you've already bought into a tool that requires the library as
> standard, say).
As much as we would like to, we have not changed the default compression
for pgBackRest. lz4 and zstd are both optional at build time since they
are not always available (though lz4 is getting close). So we have left
gzip as the default, though a lot of that has to do with maintaining
compatibility with prior versions of pgBackRest.
Having said all that, I'm absolutely in favor of adding zstd to Postgres
as an optional compression format.
Regards,
David
view thread (9+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: adding 'zstd' as a compression algorithm
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox