pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Server-side gzip compression.
6+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2022-01-27 71cbbbbe80 pg_basebackup: Add a dummy return to bbsink_gzip_new().
* pgsql: Server-side gzip compression.
@ 2022-01-24 20:14 Robert Haas <rhaas@postgresql.org>
0 siblings, 1 reply; 6+ messages in thread
From: Robert Haas @ 2022-01-24 20:14 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Server-side gzip compression.
pg_basebackup's --compression option now lets you write either
"client-gzip" or "server-gzip" instead of just "gzip" to specify
where the compression should be performed. If you write simply
"gzip" it's taken to mean "client-gzip" unless you also use
--target, in which case it is interpreted to mean "server-gzip",
because that's the only thing that makes any sense in that case.
To make this work, the BASE_BACKUP command now takes new
COMPRESSION and COMPRESSION_LEVEL options.
At present, pg_basebackup cannot decompress .gz files, so
server-side compression will cause a failure if (1) -Ft is not
used or (2) -R is used or (3) -D- is used without --no-manifest.
Along the way, I removed the information message added by commit
5c649fe153367cdab278738ee4aebbfd158e0546 which occurred if you
specified no compression level and told you that the default level
had been used instead. That seemed like more output than most
people would want.
Also along the way, this adds a check to the server for
unrecognized base backup options. This repairs a bug introduced
by commit 0ba281cb4bf9f5f65529dfa4c8282abb734dd454.
This commit also adds some new test cases for pg_verifybackup.
They take a server-side backup with and without compression, and
then extract the backup if we have the OS facilities available
to do so, and then run pg_verifybackup on the extracted
directory. That is a good test of the functionality added by
this commit and also improves test coverage for the backup target
patch (commit 3500ccc39b0dadd1068a03938e4b8ff562587ccc) and for
pg_verifybackup itself.
Patch by me, with a bug fix by Jeevan Ladhe. The patch set of which
this is a part has also had review and/or testing from Tushar Ahuja,
Suraj Kharage, Dipesh Pandit, and Mark Dilger.
Discussion: http://postgr.es/m/CA+Tgmoa-ST7fMLsVJduOB7Eub=2WjfpHS+QxHVEpUoinf4bOSg@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/0ad8032910d5eb8efd32867c45b6a25c85e60f50
Modified Files
--------------
doc/src/sgml/protocol.sgml | 22 +++
doc/src/sgml/ref/pg_basebackup.sgml | 29 ++-
src/backend/Makefile | 2 +-
src/backend/replication/Makefile | 1 +
src/backend/replication/basebackup.c | 54 ++++++
src/backend/replication/basebackup_gzip.c | 309 ++++++++++++++++++++++++++++++
src/bin/pg_basebackup/pg_basebackup.c | 136 +++++++++++--
src/bin/pg_verifybackup/Makefile | 7 +
src/bin/pg_verifybackup/t/008_untar.pl | 104 ++++++++++
src/include/replication/basebackup_sink.h | 1 +
10 files changed, 641 insertions(+), 24 deletions(-)
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgsql: Server-side gzip compression.
@ 2022-01-25 09:19 David Rowley <dgrowleyml@gmail.com>
parent: Robert Haas <rhaas@postgresql.org>
0 siblings, 1 reply; 6+ messages in thread
From: David Rowley @ 2022-01-25 09:19 UTC (permalink / raw)
To: Robert Haas <rhaas@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org
On Tue, 25 Jan 2022 at 09:14, Robert Haas <rhaas@postgresql.org> wrote:
> src/backend/replication/basebackup_gzip.c | 309 ++++++++++++++++++++++++++++++
This could do with the attached. MSVC compilers need a bit more
reassurance that ereport/elog ERRORs don't return.
David
Attachments:
fix_basebackup_gzip_compiler_warning.patch (application/octet-stream, 500B) download
diff --git a/src/backend/replication/basebackup_gzip.c b/src/backend/replication/basebackup_gzip.c
index 1e58382fa0..b66d3da7a3 100644
--- a/src/backend/replication/basebackup_gzip.c
+++ b/src/backend/replication/basebackup_gzip.c
@@ -66,6 +66,7 @@ bbsink_gzip_new(bbsink *next, int compresslevel)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("gzip compression is not supported by this build")));
+ return NULL; /* keep compiler quiet */
#else
bbsink_gzip *sink;
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgsql: Server-side gzip compression.
@ 2022-01-25 18:12 Robert Haas <robertmhaas@gmail.com>
parent: David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: Robert Haas @ 2022-01-25 18:12 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Jan 25, 2022 at 4:20 AM David Rowley <dgrowleyml@gmail.com> wrote:
> On Tue, 25 Jan 2022 at 09:14, Robert Haas <rhaas@postgresql.org> wrote:
> > src/backend/replication/basebackup_gzip.c | 309 ++++++++++++++++++++++++++++++
>
> This could do with the attached. MSVC compilers need a bit more
> reassurance that ereport/elog ERRORs don't return.
Err, well, if we need it, we need it. It surprises me, though:
wouldn't this same consideration apply to a very large number of other
places in the code base?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgsql: Server-side gzip compression.
@ 2022-01-25 20:56 David Rowley <dgrowleyml@gmail.com>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: David Rowley @ 2022-01-25 20:56 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, 26 Jan 2022 at 07:12, Robert Haas <robertmhaas@gmail.com> wrote:
> wouldn't this same consideration apply to a very large number of other
> places in the code base?
All of the other places are handled. See locations with "keep compiler quiet".
This one is the only one that generates a warning:
basebackup_gzip.c(90): warning C4715: 'bbsink_gzip_new': not all
control paths return a value
David
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgsql: Server-side gzip compression.
@ 2022-01-27 19:43 Robert Haas <robertmhaas@gmail.com>
parent: David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: Robert Haas @ 2022-01-27 19:43 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Jan 25, 2022 at 3:56 PM David Rowley <dgrowleyml@gmail.com> wrote:
> On Wed, 26 Jan 2022 at 07:12, Robert Haas <robertmhaas@gmail.com> wrote:
> > wouldn't this same consideration apply to a very large number of other
> > places in the code base?
>
> All of the other places are handled. See locations with "keep compiler quiet".
>
> This one is the only one that generates a warning:
>
> basebackup_gzip.c(90): warning C4715: 'bbsink_gzip_new': not all
> control paths return a value
OK. I'm still surprised, but it is what it is. I've committed this now.
FWIW, I would have been fine with you just committing this change. I
can't see the warning locally, so I'm not in a position to
second-guess your statement that it's needed.
Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pgsql: Server-side gzip compression.
@ 2022-01-31 04:07 David Rowley <dgrowleyml@gmail.com>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 6+ messages in thread
From: David Rowley @ 2022-01-31 04:07 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, 28 Jan 2022 at 08:44, Robert Haas <robertmhaas@gmail.com> wrote:
> OK. I'm still surprised, but it is what it is. I've committed this now.
Thanks
> FWIW, I would have been fine with you just committing this change.
That's good to know, thanks for mentioning it. FWIW, I just held back
as I wasn't 100% sure on the etiquette.
David
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2022-01-31 04:07 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-01-24 20:14 pgsql: Server-side gzip compression. Robert Haas <rhaas@postgresql.org>
2022-01-25 09:19 ` David Rowley <dgrowleyml@gmail.com>
2022-01-25 18:12 ` Robert Haas <robertmhaas@gmail.com>
2022-01-25 20:56 ` David Rowley <dgrowleyml@gmail.com>
2022-01-27 19:43 ` Robert Haas <robertmhaas@gmail.com>
2022-01-31 04:07 ` David Rowley <dgrowleyml@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox