From: Chapman Flack <chap@anastigmatix.net>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Michael Paquier <michael.paquier@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: AdvanceXLInsertBuffer vs. WAL segment compressibility
Date: Tue, 26 Jul 2016 09:31:50 -0400
Message-ID: <57976646.70502@anastigmatix.net> (raw)
In-Reply-To: <CAA4eK1K_qmdeVoitjY0vyLcEBRu=o5Mwqz-+6hA+nG=KnS_kHA@mail.gmail.com>
References: <579297F8.7020107@anastigmatix.net>
<CAA4eK1L7UvOS8TpUvoMuGZ4Mid=Mef-Cq5Q0M-2oYSHC-CSikw@mail.gmail.com>
<57962061.9010506@anastigmatix.net>
<CAB7nPqQ5NS5E-rWzd76MQHBEoc5cGHjSrUs4W6EiTxwWEx338Q@mail.gmail.com>
<5796D9EB.6070407@anastigmatix.net>
<CAA4eK1K_qmdeVoitjY0vyLcEBRu=o5Mwqz-+6hA+nG=KnS_kHA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 07/26/2016 08:48 AM, Amit Kapila wrote:
> general, if you have a very low WAL activity, then the final size of
> compressed WAL shouldn't be much even if you use gzip. It seems your
9.5 pg_xlog, low activity test cluster (segment switches forced
only by checkpoint timeouts), compressed with gzip -9:
$ for i in 0*; do echo -n "$i " && gzip -9 <$i | wc -c; done
000000010000000100000042 27072
000000010000000100000043 27075
000000010000000100000044 27077
000000010000000100000045 27073
000000010000000100000046 27075
Log from live pre-9.4 cluster, low-activity time of day, delta
compression using rsync:
2016-07-26 03:54:02 EDT (walship) INFO: using 2.39s user, 0.4s system,
9.11s on
wall:
231 byte 000000010000004600000029_000000010000004600000021_fwd
...
2016-07-26 04:54:01 EDT (walship) INFO: using 2.47s user, 0.4s system,
8.43s on
wall:
232 byte 00000001000000460000002A_000000010000004600000022_fwd
...
2016-07-26 05:54:02 EDT (walship) INFO: using 2.56s user, 0.29s system,
9.44s on
wall:
230 byte 00000001000000460000002B_000000010000004600000023_fwd
So when I say "factor of 100", I'm understating slightly. (Those
timings, for the curious, include sending a copy offsite via ssh.)
> everything zero. Now, it might be possible to selectively initialize
> the fields that doesn't harm the methodology for archive you are using
> considering there is no other impact of same in code. However, it
Indeed, it is only the one header field that duplicates the low-
order part of the (hex) file name that breaks delta compression,
because it has always been incremented even when nothing else is
different, and it's scattered 2000 times through the file.
Would it break anything for *that* to be zero in dummy blocks?
-Chap
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: chap@anastigmatix.net, amit.kapila16@gmail.com, michael.paquier@gmail.com
Subject: Re: AdvanceXLInsertBuffer vs. WAL segment compressibility
In-Reply-To: <57976646.70502@anastigmatix.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox