Subject: Re: AdvanceXLInsertBuffer vs. WAL segment compressibility
To: Heikki Linnakangas <hlinnaka@iki.fi>, Robert Haas
 <robertmhaas@gmail.com>, Bruce Momjian <bruce@momjian.us>
References: <579297F8.7020107@anastigmatix.net>
 <CA+TgmoaJR3s6CymT2wa9L5QR0pKY0cBthfece8MtxaBADMXw0Q@mail.gmail.com>
 <5797D953.6040806@anastigmatix.net> <20160802183347.GD32575@momjian.us>
 <CA+TgmobKFW5eVx4gd9TyiXv9iL_5DcC0-6UuJLXvwG1Ft1Pn-g@mail.gmail.com>
 <20943eba-bbb9-390c-01b1-e6704ce33c7b@iki.fi>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
From: Chapman Flack <chap@anastigmatix.net>
Message-ID: <594C9457.6000708@anastigmatix.net>
Date: Fri, 23 Jun 2017 00:08:55 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.7.0
MIME-Version: 1.0
In-Reply-To: <20943eba-bbb9-390c-01b1-e6704ce33c7b@iki.fi>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 06/21/17 04:51, Heikki Linnakangas wrote:
> (I'm cleaning up my inbox, hence the delayed reply)

I had almost completely forgotten ever bringing it up. :)

> When I wrote that code, I don't remember if I realized that we're
> initializing the page headers, or if I thought that it's good enough even if
> we do. I guess I didn't realize it, because a comment would've been in order
> if it was intentional.
> 
> So +1 on fixing that, a patch would be welcome.

Ok, that sounds like something I could take a whack at. Overall, xlog.c
is a bit daunting, but this particular detail seems fairly approachable.

> In the meanwhile, have you
> tried using a different compression program? Something else than gzip might
> do a better job at the almost zero pages.

Well, gzip was doing pretty well; it could get a 16 MB segment file down
to under 27 kB, or less than 14 bytes for each of 2000 pages, when a page
header is what, 20 bytes, it looks like? I'm not sure how much better
I'd expect a (non-custom) compression scheme to do. The real difference
comes between compressing (even well) a large unchanged area, versus being
able to recognize (again with a non-custom tool) that the whole area is
unchanged.

-Chap


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers