From: Tom Lane <tgl@sss.pgh.pa.us>
To: Euler Taveira <euler@timbira.com>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Subject: Re: libpq compression
Date: Thu, 14 Jun 2012 01:19:30 -0400
Message-ID: <29853.1339651170@sss.pgh.pa.us> (raw)
In-Reply-To: <4FD9698F.2090407@timbira.com>
References: <4FD9698F.2090407@timbira.com>
Euler Taveira <euler@timbira.com> writes:
> There was already some discussion about compressing libpq data [1][2][3].
> Recently, I faced a scenario that would become less problematic if we have had
> compression support. The scenario is frequent data load (aka COPY) over
> slow/unstable links. It should be executed in a few hundreds of PostgreSQL
> servers all over Brazil. Someone could argue that I could use ssh tunnel to
> solve the problem but (i) it is complex because it involves a different port
> in the firewall and (ii) it's an opportunity to improve other scenarios like
> reducing bandwidth consumption during replication or normal operation over
> slow/unstable links.
I still think that pushing this off to openssl (not an ssh tunnel, but
the underlying transport library) would be an adequate solution.
If you are shoving data over a connection that is long enough to need
compression, the odds that every bit of it is trustworthy seem pretty
small, so you need encryption too.
We do need the ability to tell openssl to use compression. We don't
need to implement it ourselves, nor to bring a bunch of new library
dependencies into our builds. I especially think that importing bzip2
is a pretty bad idea --- it's not only a new dependency, but bzip2's
compression versus speed tradeoff is entirely inappropriate for this
use-case.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us, euler@timbira.com
Subject: Re: libpq compression
In-Reply-To: <29853.1339651170@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox