From: Ildus K <i.kurbangaliev@postgrespro.ru>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Custom compression methods
Date: Tue, 21 Nov 2017 23:28:55 +0300
Message-ID: <20171121232855.349e14a9@postgrespro.ru> (raw)
In-Reply-To: <bfb3d34e-ba74-02cb-ca2a-43344866d9fd@2ndquadrant.com>
References: <20170907194236.4cefce96@wp.localdomain>
<20170912175505.4afa11fd@wp.localdomain>
<20171102152836.60c041e4@wp.localdomain>
<20171114162356.52e3d388@wp.localdomain>
<62e46a47-08a8-6e06-5e4f-6f52e0a13202@2ndquadrant.com>
<20171121174717.69ecd8f4@wp.localdomain>
<bfb3d34e-ba74-02cb-ca2a-43344866d9fd@2ndquadrant.com>
On Tue, 21 Nov 2017 18:47:49 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
>
> I propose to use either
>
> CompressionMethodOptions (and CompressionMethodRoutine)
>
> or
>
> CompressionOptions (and CompressionRoutine)
Sounds good, thanks.
>
> OK. But then I don't understand why tsvector.c does things like
>
> VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize
> VARRAWSIZE_4B_C(data) - arrsize
>
> instead of
>
> VARSIZE_ANY_EXHDR(data) - arrsize
> VARSIZE_ANY(data) - arrsize
>
> Seems somewhat confusing.
>
VARRAWSIZE_4B_C returns original size of data, before compression (from
va_rawsize in current postgres, and from va_info in my patch), not size
of the already compressed data, so you can't use VARSIZE_ANY here.
VARSIZE_ANY_EXHDR in current postgres returns VARSIZE-VARHDRSZ, despite
the varlena is compressed or not, so I just kept this behavior for
custom compressed varlenas too. If you look into tuptoaster.c you will
also see lines like 'VARSIZE(attr) - TOAST_COMPRESS_HDRSZ'. So I think
if VARSIZE_ANY_EXHDR will subtract different header sizes then it
should subtract them for usual compressed varlenas too.
> >
>
> Hmmm, it still doesn't work for me. See this:
>
> test=# create extension pg_lz4 ;
> CREATE EXTENSION
> test=# create table t_lz4 (v text compressed lz4);
> CREATE TABLE
> test=# create table t_pglz (v text);
> CREATE TABLE
> test=# insert into t_lz4 select repeat(md5(1::text),300);
> INSERT 0 1
> test=# insert into t_pglz select * from t_lz4;
> INSERT 0 1
> test=# drop extension pg_lz4 cascade;
> NOTICE: drop cascades to 2 other objects
> DETAIL: drop cascades to compression options for lz4
> drop cascades to table t_lz4 column v
> DROP EXTENSION
> test=# \c test
> You are now connected to database "test" as user "user".
> test=# insert into t_lz4 select repeat(md5(1::text),300);^C
> test=# select * from t_pglz ;
> ERROR: cache lookup failed for compression options 16419
>
> That suggests no recompression happened.
I will check that. Is your extension published somewhere?
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: i.kurbangaliev@postgrespro.ru, tomas.vondra@2ndquadrant.com
Subject: Re: [HACKERS] Custom compression methods
In-Reply-To: <20171121232855.349e14a9@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox