Date: Mon, 20 Nov 2017 12:44:28 +0300
From: Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>, Andres Freund
 <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Oleg Bartunov
 <obartunov@gmail.com>, Craig Ringer <craig@2ndquadrant.com>, Peter
 Eisentraut <peter.eisentraut@2ndquadrant.com>, PostgreSQL Hackers
 <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Custom compression methods
Message-ID: <20171120124428.6154f23f@wp.localdomain>
In-Reply-To: <e13abfa9-658d-8bef-4a69-68ed4ea34ee9@2ndquadrant.com>
References: <20170907194236.4cefce96@wp.localdomain>
	<20170912175505.4afa11fd@wp.localdomain>
	<5c84382f-1065-e9e8-dde8-4c1f5ae1007b@2ndquadrant.com>
	<20171102124101.5a28ecab@wp.localdomain>
	<CAMsr+YGm0z5571OmwyPaq94A03MaTfJq4PfR-r=uUpuidSTjeA@mail.gmail.com>
	<CAF4Au4zp-S1srXbQD5XKHFaOky+vVY5dPAUgCi8Ziyp9sTTRWg@mail.gmail.com>
	<CA+TgmoZZYX-knZoTojo-dTbTvmRw8fzFQzH=MXs9GDN3m9-QeQ@mail.gmail.com>
	<20171115120928.31bee414@wp.localdomain>
	<CA+TgmoachY0-FjthGkpEDTswt3i7h1xfMzjow25d5NNuvaaw7g@mail.gmail.com>
	<e13abfa9-658d-8bef-4a69-68ed4ea34ee9@2ndquadrant.com>
Organization: Postgres Professional
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Mon, 20 Nov 2017 00:23:23 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:

> On 11/15/2017 02:13 PM, Robert Haas wrote:
> > On Wed, Nov 15, 2017 at 4:09 AM, Ildus Kurbangaliev
> > <i.kurbangaliev@postgrespro.ru> wrote:  
> >> So in the next version of the patch I can just unlink the options
> >> from compression methods and dropping compression method will not
> >> affect already compressed tuples. They still could be
> >> decompressed.  
> > 
> > I guess I don't understand how that can work.  I mean, if somebody
> > removes a compression method - i.e. uninstalls the library - and you
> > don't have a way to make sure there are no tuples that can only be
> > uncompressed by that library - then you've broken the database.
> > Ideally, there should be a way to add a new compression method via
> > an extension ... and then get rid of it and all dependencies
> > thereupon. 
> 
> I share your confusion. Once you do DROP COMPRESSION METHOD, there
> must be no remaining data compressed with it. But that's what the
> patch is doing already - it enforces this using dependencies, as
> usual.
> 
> Ildus, can you explain what you meant? How could the data still be
> decompressed after DROP COMPRESSION METHOD, and possibly after
> removing the .so library?

The removal of the .so library will broke all compressed tuples. I
don't see a way to avoid it. I meant that DROP COMPRESSION METHOD could
remove the record from 'pg_compression' table, but actually the
compressed tuple needs only a record from 'pg_compression_opt' where
its options are located. And there is dependency between an extension
and the options so you can't just remove the extension without CASCADE,
postgres will complain.

Still it's a problem if the user used for example `SELECT
<compressed_column> INTO * FROM *` because postgres will copy compressed
tuples, and there will not be any dependencies between destination and
the options.

Also thank you for review. I will look into it today.


-- 
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company