Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eH9pm-0006nW-1o for pgsql-hackers@arkaria.postgresql.org; Tue, 21 Nov 2017 14:47:35 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eH9pl-0007Hq-Je for pgsql-hackers@arkaria.postgresql.org; Tue, 21 Nov 2017 14:47:33 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1eH9pk-0007Hg-LP for pgsql-hackers@lists.postgresql.org; Tue, 21 Nov 2017 14:47:33 +0000 Received: from mail.postgrespro.ru ([93.174.131.138]) by makus.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1eH9pa-0002yC-IT for pgsql-hackers@postgresql.org; Tue, 21 Nov 2017 14:47:31 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.postgrespro.ru (Postfix) with ESMTP id 3E30C21C1CCF; Tue, 21 Nov 2017 17:47:19 +0300 (MSK) X-Virus-Scanned: Debian amavisd-new at postgrespro.ru Received: from wp.localdomain (gw.postgrespro.ru [93.174.131.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id B428D21C1756; Tue, 21 Nov 2017 17:47:18 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1511275638; bh=SLwDL7P0mZ0qfpspI9Cvlwkj3/MW3M49P9/+qfooZqM=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=erP/1KLn0532mHIw6u6ET17QxYPuPYiFfi5yHgb4BMAwwOFZxtqn8JFN9hbl37Y9+ FcdxX/TyL9cFDnp5Ycb2MRAPcei7Y3kpGAQKtykfojRF0znQGQm+98ve2iGK1cwZW2 wqNDFiFSjZotPTZD5JUCaQ0XAEDKZ5OIN8EWRNSk= Date: Tue, 21 Nov 2017 17:47:17 +0300 From: Ildus Kurbangaliev To: Tomas Vondra Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Custom compression methods Message-ID: <20171121174717.69ecd8f4@wp.localdomain> In-Reply-To: <62e46a47-08a8-6e06-5e4f-6f52e0a13202@2ndquadrant.com> References: <20170907194236.4cefce96@wp.localdomain> <20170912175505.4afa11fd@wp.localdomain> <20171102152836.60c041e4@wp.localdomain> <20171114162356.52e3d388@wp.localdomain> <62e46a47-08a8-6e06-5e4f-6f52e0a13202@2ndquadrant.com> Organization: Postgres Professional X-Mailer: Claws Mail 3.15.1-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/QRJJj=Wu9Xyc77uyK8Zsh.S" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --MP_/QRJJj=Wu9Xyc77uyK8Zsh.S Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline On Mon, 20 Nov 2017 00:04:53 +0100 Tomas Vondra wrote: > I did a review of this today, and I think there are some things that > need improvement / fixing. > > Firstly, some basic comments from just eye-balling the diff, then some > bugs I discovered after writing an extension adding lz4. > > 1) formatRelOptions/freeRelOptions are no longer needed (I see Ildar > already pointer that out) I removed freeRelOptions, but formatRelOptions is used in other place. > > 2) There's unnecessary whitespace (extra newlines) on a couple of > places, which is needlessly increasing the size of the patch. Small > difference, but annoying. Cleaned up. > > 3) tuptoaster.c > > Why do you change 'info' from int32 to uint32? Seems unnecessary. That's because I use highest bit, and it makes number negative for int32. I use right shifting to get that bit and right shift on negative gives negative value too. > > Adding new 'att' variable in toast_insert_or_update is confusing, as > there already is 'att' in the very next loop. Technically it's > correct, but I'd bet it'll lead to some WTF?! moments later. I > propose to just use TupleDescAttr(tupleDesc,i) on the two places > where it matters, around line 808. > > There are no comments for init_compression_options_htab and > get_compression_options_info, so that needs to be fixed. Moreover, the > names are confusing because what we really get is not just 'options' > but the compression routines too. Removed extra 'att', and added comments. > > 4) gen_db_file_maps probably shouldn't do the fprints, right? > > 5) not sure why you modify src/tools/pgindent/exclude_file_patterns My bad, removed these lines. > > 6) I'm rather confused by AttributeCompression vs. ColumnCompression. > I mean, attribute==column, right? Of course, one is for data from > parser, the other one is for internal info. But can we make the > naming clearer? For now I have renamed AttributeCompression to CompressionOptions, not sure that's a good name but at least it gives less confusion. > > 7) The docs in general are somewhat unsatisfactory, TBH. For example > the ColumnCompression has no comments, unlike everything else in > parsenodes. Similarly for the SGML docs - I suggest to expand them to > resemble FDW docs > (https://www.postgresql.org/docs/10/static/fdwhandler.html) which > also follows the handler/routines pattern. I've added more comments. I think I'll add more documentation if the committers will approve current syntax. > > 8) One of the unclear things if why we even need 'drop' routing. It > seems that if it's defined DropAttributeCompression does something. > But what should it do? I suppose dropping the options should be done > using dependencies (just like we drop columns in this case). > > BTW why does DropAttributeCompression mess with att->attisdropped in > this way? That seems a bit odd. 'drop' routine could be useful. An extension could do something related with the attribute, like remove extra tables or something else. The compression options will not be removed after unlinking compression method from a column because there is still be stored compressed data in that column. That 'attisdropped' part has been removed. > > 9) configure routines that only check if (options != NIL) and then > error out (like tsvector_configure) seem a bit unnecessary. Just > allow it to be NULL in CompressionMethodRoutine, and throw an error > if options is not NIL for such compression method. Good idea, done. > > 10) toast_compress_datum still does this: > > if (!ac && (valsize < PGLZ_strategy_default->min_input_size || > valsize > PGLZ_strategy_default->max_input_size)) > > which seems rather pglz-specific (the naming is a hint). Why shouldn't > this be specific to compression, exposed either as min/max constants, > or wrapped in another routine - size_is_valid() or something like > that? I agree, moved to the next block related with pglz. > > 11) The comments in toast_compress_datum probably need updating, as it > still references to pglz specifically. I guess the new compression > methods do matter too. Done. > > 12) get_compression_options_info organizes the compression info into a > hash table by OID. The hash table implementation assumes the hash key > is at the beginning of the entry, but AttributeCompression is defined > like this: > > typedef struct > { > CompressionMethodRoutine *routine; > List *options; > Oid cmoptoid; > } AttributeCompression; > > Which means get_compression_options_info is busted, will never lookup > anything, and the hash table will grow by adding more and more entries > into the same bucket. Of course, this has extremely negative impact on > performance (pretty much arbitrarily bad, depending on how many > entries you've already added to the hash table). > > Moving the OID to the beginning of the struct fixes the issue. Yeah, I fixed it before, but somehow managed to do not include it to the patch. > > 13) When writing the experimental extension, I was extremely confused > about the regular varlena headers, custom compression headers, etc. In > the end I stole the code from tsvector.c and whacked it a bit until it > worked, but I wouldn't dare to claim I understand how it works. > > This needs to be documented somewhere. For example postgres.h has a > bunch of paragraphs about varlena headers, so perhaps it should be > there? I see the patch tweaks some of the constants, but does not > update the comment at all. This point is good, I'm not sure how this documentation should look like. I've just assumed that people should have deep undestanding of varlenas if they're going to compress them. But now it's easy to make mistake there. Maybe I should add some functions that help to construct varlena, with different headers. I like the way is how jsonb is constructed. It uses StringInfo and there are few helper functions (reserveFromBuffer, appendToBuffer and others). Maybe they should be not static. > > Perhaps it would be useful to provide some additional macros making > access to custom-compressed varlena values easier. Or perhaps the > VARSIZE_ANY / VARSIZE_ANY_EXHDR / VARDATA_ANY already support that? > This part is not very clear to me. These macros will work, custom compressed varlenas behave like old compressed varlenas. > > Still it's a problem if the user used for example `SELECT > > INTO * FROM *` because postgres will copy > > compressed tuples, and there will not be any dependencies between > > destination and the options. > > > > This seems like a rather fatal design flaw, though. I'd say we need to > force recompression of the data, in such cases. Otherwise all the > dependency tracking is rather pointless. Fixed this problem too. I've added recompression for datum that use custom compression. -- --- Ildus Kurbangaliev Postgres Professional: http://www.postgrespro.com Russian Postgres Company --MP_/QRJJj=Wu9Xyc77uyK8Zsh.S Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=custom_compression_methods_v5.patch diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out index 1e22c1eefc..766ced401f 100644 --- a/contrib/test_decoding/expected/ddl.out +++ b/contrib/test_decoding/expected/ddl.out @@ -416,12 +416,12 @@ CREATE TABLE replication_metadata ( WITH (user_catalog_table = true) ; \d+ replication_metadata - Table "public.replication_metadata" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description -----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | - relation | name | | not null | | plain | | - options | text[] | | | | extended | | + Table "public.replication_metadata" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | | + relation | name | | not null | | plain | | | + options | text[] | | | | extended | | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) Options: user_catalog_table=true @@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options) VALUES ('foo', ARRAY['a', 'b']); ALTER TABLE replication_metadata RESET (user_catalog_table); \d+ replication_metadata - Table "public.replication_metadata" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description -----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | - relation | name | | not null | | plain | | - options | text[] | | | | extended | | + Table "public.replication_metadata" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | | + relation | name | | not null | | plain | | | + options | text[] | | | | extended | | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) @@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options) VALUES ('bar', ARRAY['a', 'b']); ALTER TABLE replication_metadata SET (user_catalog_table = true); \d+ replication_metadata - Table "public.replication_metadata" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description -----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | - relation | name | | not null | | plain | | - options | text[] | | | | extended | | + Table "public.replication_metadata" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | | + relation | name | | not null | | plain | | | + options | text[] | | | | extended | | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) Options: user_catalog_table=true @@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text; ERROR: cannot rewrite table "replication_metadata" used as a catalog table ALTER TABLE replication_metadata SET (user_catalog_table = false); \d+ replication_metadata - Table "public.replication_metadata" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description -----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | - relation | name | | not null | | plain | | - options | text[] | | | | extended | | - rewritemeornot | integer | | | | plain | | + Table "public.replication_metadata" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | | + relation | name | | not null | | plain | | | + options | text[] | | | | extended | | | + rewritemeornot | integer | | | | plain | | | Indexes: "replication_metadata_pkey" PRIMARY KEY, btree (id) Options: user_catalog_table=false diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml index 01acc2ef9d..f43f09cc19 100644 --- a/doc/src/sgml/ref/allfiles.sgml +++ b/doc/src/sgml/ref/allfiles.sgml @@ -59,6 +59,7 @@ Complete list of usable sgml source files in this directory. + diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 92db00f52d..207d6cf26e 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -53,6 +53,8 @@ ALTER TABLE [ IF EXISTS ] name ALTER [ COLUMN ] column_name SET ( attribute_option = value [, ... ] ) ALTER [ COLUMN ] column_name RESET ( attribute_option [, ... ] ) ALTER [ COLUMN ] column_name SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN } + ALTER [ COLUMN ] column_name SET COMPRESSED compression_method_name [ WITH (compression_method_options) ] + ALTER [ COLUMN ] column_name SET NOT COMPRESSED ADD table_constraint [ NOT VALID ] ADD table_constraint_using_index ALTER CONSTRAINT constraint_name [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] @@ -320,6 +322,34 @@ ALTER TABLE [ IF EXISTS ] name + + + SET COMPRESSED compression_method_name + + + + This form adds compression to a column. Compression method should be + created with . If compression + method has options they could be specified with WITH + parameter. Setting a compression method doesn't change anything in the + table and affects only future table updates. + + + + + + + SET NOT COMPRESSED + + + + This form removes compression from a column. Removing compresssion from + a column doesn't change already compressed tuples and affects only future + table updates. + + + + ADD table_constraint [ NOT VALID ] diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml new file mode 100644 index 0000000000..663010ecd9 --- /dev/null +++ b/doc/src/sgml/ref/create_compression_method.sgml @@ -0,0 +1,50 @@ + + + + + CREATE COMPRESSION METHOD + + + + CREATE COMPRESSION METHOD + 7 + SQL - Language Statements + + + + CREATE COMPRESSION METHOD + define a new compression method + + + + +CREATE COMPRESSION METHOD compression_method_name + HANDLER compression_method_handler + + + + + Description + + + CREATE COMPRESSION METHOD creates a new compression method + with compression_method_name. + + + + A compression method links a name with a compression handler. And the + handler is a special function that returns collection of methods that + can be used for compression. + + + + After a compression method is created, you can specify it in + or + statements. + + + + diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 83eef7f10d..9631279cb0 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( sequence_options ) ] | UNIQUE index_parameters | PRIMARY KEY index_parameters | + COMPRESSED compression_method_name [ WITH (compression_method_options) ] | REFERENCES reftable [ ( refcolumn ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] [ ON DELETE action ] [ ON UPDATE action ] } [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] @@ -817,6 +818,18 @@ WITH ( MODULUS numeric_literal, REM + + COMPRESSED compression_method_name [ WITH (compression_method_options) ] + + + This clause adds compression to a column. Compression method should be + created with . If compression + method has options they could be specified with WITH + parameter. + + + + EXCLUDE [ USING index_method ] ( exclude_element WITH operator [, ... ] ) index_parameters [ WHERE ( predicate ) ] diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml index 9000b3aaaa..cc0bd70be3 100644 --- a/doc/src/sgml/reference.sgml +++ b/doc/src/sgml/reference.sgml @@ -87,6 +87,7 @@ &createAggregate; &createCast; &createCollation; + &createCompressionMethod; &createConversion; &createDatabase; &createDomain; diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c index 138671410a..629f3575a9 100644 --- a/src/backend/access/common/indextuple.c +++ b/src/backend/access/common/indextuple.c @@ -92,7 +92,8 @@ index_form_tuple(TupleDesc tupleDescriptor, VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET && (att->attstorage == 'x' || att->attstorage == 'm')) { - Datum cvalue = toast_compress_datum(untoasted_values[i]); + Datum cvalue = toast_compress_datum(untoasted_values[i], + att->attcompression); if (DatumGetPointer(cvalue) != NULL) { diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c index aa9c0f1bb9..f740ce4304 100644 --- a/src/backend/access/common/reloptions.c +++ b/src/backend/access/common/reloptions.c @@ -945,11 +945,31 @@ untransformRelOptions(Datum options) val = (Node *) makeString(pstrdup(p)); } result = lappend(result, makeDefElem(pstrdup(s), val, -1)); + pfree(s); } return result; } +char * +formatRelOptions(List *options) +{ + StringInfoData buf; + ListCell *cell; + + initStringInfo(&buf); + + foreach(cell, options) + { + DefElem *def = (DefElem *) lfirst(cell); + + appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "", + def->defname, defGetString(def)); + } + + return buf.data; +} + /* * Extract and parse reloptions from a pg_class tuple. * diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 9e37ca73a8..d206cce18e 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -19,8 +19,10 @@ #include "postgres.h" +#include "access/compression.h" #include "access/hash.h" #include "access/htup_details.h" +#include "access/reloptions.h" #include "catalog/pg_collation.h" #include "catalog/pg_type.h" #include "miscadmin.h" @@ -242,6 +244,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno, dstAtt->attnotnull = false; dstAtt->atthasdef = false; dstAtt->attidentity = '\0'; + dstAtt->attcompression = InvalidOid; } /* @@ -396,6 +399,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2) return false; if (attr1->attcollation != attr2->attcollation) return false; + if (attr1->attcompression != attr2->attcompression) + return false; /* attacl, attoptions and attfdwoptions are not even present... */ } @@ -458,6 +463,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2) } else if (tupdesc2->constr != NULL) return false; + return true; } @@ -563,6 +569,7 @@ TupleDescInitEntry(TupleDesc desc, att->attalign = typeForm->typalign; att->attstorage = typeForm->typstorage; att->attcollation = typeForm->typcollation; + att->attcompression = InvalidOid; ReleaseSysCache(tuple); } @@ -675,7 +682,6 @@ TupleDescInitEntryCollation(TupleDesc desc, TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid; } - /* * BuildDescForRelation * diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c index c74945a52a..99307cb5e6 100644 --- a/src/backend/access/heap/tuptoaster.c +++ b/src/backend/access/heap/tuptoaster.c @@ -30,8 +30,10 @@ #include #include +#include "access/compression.h" #include "access/genam.h" #include "access/heapam.h" +#include "access/reloptions.h" #include "access/tuptoaster.h" #include "access/xact.h" #include "catalog/catalog.h" @@ -39,6 +41,8 @@ #include "miscadmin.h" #include "utils/expandeddatum.h" #include "utils/fmgroids.h" +#include "utils/hsearch.h" +#include "utils/memutils.h" #include "utils/rel.h" #include "utils/snapmgr.h" #include "utils/typcache.h" @@ -53,19 +57,52 @@ typedef struct toast_compress_header { int32 vl_len_; /* varlena header (do not touch directly!) */ - int32 rawsize; + uint32 info; /* flags (2 bits) and rawsize */ } toast_compress_header; +/* + * If the compression method were used, then data also contains + * Oid of compression options + */ +typedef struct toast_compress_header_custom +{ + int32 vl_len_; /* varlena header (do not touch directly!) */ + uint32 info; /* flags (2 high bits) and rawsize */ + Oid cmoptoid; /* Oid from pg_compression_opt */ +} toast_compress_header_custom; + +static HTAB *compression_options_cache = NULL; +static MemoryContext compression_options_mcxt = NULL; + +#define RAWSIZEMASK (0x3FFFFFFFU) + /* * Utilities for manipulation of header information for compressed * toast entries. */ -#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header)) -#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize) +#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header)) +#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom)) +#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK) #define TOAST_COMPRESS_RAWDATA(ptr) \ (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ) #define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \ - (((toast_compress_header *) (ptr))->rawsize = (len)) +do { \ + ((toast_compress_header *) (ptr))->info &= 0xC0000000; \ + ((toast_compress_header *) (ptr))->info |= ((uint32)(len) & RAWSIZEMASK); \ +} while (0) +#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \ + (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid)) +#define TOAST_COMPRESS_SET_CUSTOM(ptr) \ +do { \ + (((toast_compress_header *) (ptr))->info |= (1 << 31)); \ + (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \ +} while (0) + +#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \ +do { \ + ((toast_pointer).va_extinfo |= (1 << 31)); \ + ((toast_pointer).va_extinfo &= ~(1 << 30)); \ +} while (0) static void toast_delete_datum(Relation rel, Datum value, bool is_speculative); static Datum toast_save_datum(Relation rel, Datum value, @@ -83,6 +120,8 @@ static int toast_open_indexes(Relation toastrel, static void toast_close_indexes(Relation *toastidxs, int num_indexes, LOCKMODE lock); static void init_toast_snapshot(Snapshot toast_snapshot); +static void init_compression_options_cache(void); +static CompressionOptions * get_cached_compression_options(Oid cmoptoid); /* ---------- @@ -421,7 +460,7 @@ toast_datum_size(Datum value) struct varatt_external toast_pointer; VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr); - result = toast_pointer.va_extsize; + result = VARATT_EXTERNAL_GET_SIZE(toast_pointer); } else if (VARATT_IS_EXTERNAL_INDIRECT(attr)) { @@ -686,8 +725,24 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup, toast_oldexternal[i] = new_value; if (att->attstorage == 'p') new_value = heap_tuple_untoast_attr(new_value); + else if (VARATT_IS_EXTERNAL_ONDISK(new_value)) + { + struct varatt_external toast_pointer; + + VARATT_EXTERNAL_GET_POINTER(toast_pointer, new_value); + + /* + * If we're trying to insert a custom compressed datum we + * should decompress it first. + */ + if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer)) + new_value = heap_tuple_untoast_attr(new_value); + else + new_value = heap_tuple_fetch_attr(new_value); + } else new_value = heap_tuple_fetch_attr(new_value); + toast_values[i] = PointerGetDatum(new_value); toast_free[i] = true; need_change = true; @@ -741,12 +796,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup, Datum old_value; Datum new_value; + Form_pg_attribute att; + /* * Search for the biggest yet unprocessed internal attribute */ for (i = 0; i < numAttrs; i++) { - Form_pg_attribute att = TupleDescAttr(tupleDesc, i); + char attstorage; if (toast_action[i] != ' ') continue; @@ -754,7 +811,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup, continue; /* can't happen, toast_action would be 'p' */ if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i]))) continue; - if (att->attstorage != 'x' && att->attstorage != 'e') + + attstorage = (TupleDescAttr(tupleDesc, i))->attstorage; + if (attstorage != 'x' && attstorage != 'e') continue; if (toast_sizes[i] > biggest_size) { @@ -770,10 +829,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup, * Attempt to compress it inline, if it has attstorage 'x' */ i = biggest_attno; - if (TupleDescAttr(tupleDesc, i)->attstorage == 'x') + att = TupleDescAttr(tupleDesc, i); + if (att->attstorage == 'x') { old_value = toast_values[i]; - new_value = toast_compress_datum(old_value); + new_value = toast_compress_datum(old_value, att->attcompression); if (DatumGetPointer(new_value) != NULL) { @@ -914,7 +974,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup, */ i = biggest_attno; old_value = toast_values[i]; - new_value = toast_compress_datum(old_value); + new_value = toast_compress_datum(old_value, + TupleDescAttr(tupleDesc, i)->attcompression); if (DatumGetPointer(new_value) != NULL) { @@ -1353,7 +1414,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc, return new_tuple; } - /* ---------- * toast_compress_datum - * @@ -1368,41 +1428,57 @@ toast_build_flattened_tuple(TupleDesc tupleDesc, * ---------- */ Datum -toast_compress_datum(Datum value) +toast_compress_datum(Datum value, Oid cmoptoid) { - struct varlena *tmp; - int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value)); - int32 len; + struct varlena *tmp = NULL; + int32 valsize, + len = 0; + CompressionOptions *cmoptions = NULL; Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value))); Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value))); - /* - * No point in wasting a palloc cycle if value size is out of the allowed - * range for compression - */ - if (valsize < PGLZ_strategy_default->min_input_size || - valsize > PGLZ_strategy_default->max_input_size) - return PointerGetDatum(NULL); + if (OidIsValid(cmoptoid)) + cmoptions = get_cached_compression_options(cmoptoid); - tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) + - TOAST_COMPRESS_HDRSZ); + valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value)); + + if (cmoptions) + { + tmp = cmoptions->routine->compress(cmoptions, (const struct varlena *) value); + if (!tmp) + return PointerGetDatum(NULL); + } + else + { + /* + * No point in wasting a palloc cycle if value size is out of the + * allowed range for compression + */ + if (valsize < PGLZ_strategy_default->min_input_size || + valsize > PGLZ_strategy_default->max_input_size) + return PointerGetDatum(NULL); + + tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) + + TOAST_COMPRESS_HDRSZ); + len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)), + valsize, + TOAST_COMPRESS_RAWDATA(tmp), + PGLZ_strategy_default); + } /* - * We recheck the actual size even if pglz_compress() reports success, - * because it might be satisfied with having saved as little as one byte - * in the compressed data --- which could turn into a net loss once you - * consider header and alignment padding. Worst case, the compressed - * format might require three padding bytes (plus header, which is - * included in VARSIZE(tmp)), whereas the uncompressed format would take - * only one header byte and no padding if the value is short enough. So - * we insist on a savings of more than 2 bytes to ensure we have a gain. + * We recheck the actual size even if compression function reports + * success, because it might be satisfied with having saved as little as + * one byte in the compressed data --- which could turn into a net loss + * once you consider header and alignment padding. Worst case, the + * compressed format might require three padding bytes (plus header, which + * is included in VARSIZE(tmp)), whereas the uncompressed format would + * take only one header byte and no padding if the value is short enough. + * So we insist on a savings of more than 2 bytes to ensure we have a + * gain. */ - len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)), - valsize, - TOAST_COMPRESS_RAWDATA(tmp), - PGLZ_strategy_default); - if (len >= 0 && + if (!cmoptions && len >= 0 && len + TOAST_COMPRESS_HDRSZ < valsize - 2) { TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize); @@ -1410,10 +1486,20 @@ toast_compress_datum(Datum value) /* successful compression */ return PointerGetDatum(tmp); } + else if (cmoptions && VARSIZE(tmp) < valsize - 2) + { + TOAST_COMPRESS_SET_CUSTOM(tmp); + TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize); + TOAST_COMPRESS_SET_CMOPTOID(tmp, cmoptions->cmoptoid); + /* successful compression */ + return PointerGetDatum(tmp); + } else { /* incompressible data */ - pfree(tmp); + if (tmp) + pfree(tmp); + return PointerGetDatum(NULL); } } @@ -1510,19 +1596,20 @@ toast_save_datum(Relation rel, Datum value, &num_indexes); /* - * Get the data pointer and length, and compute va_rawsize and va_extsize. + * Get the data pointer and length, and compute va_rawsize and va_extinfo. * * va_rawsize is the size of the equivalent fully uncompressed datum, so * we have to adjust for short headers. * - * va_extsize is the actual size of the data payload in the toast records. + * va_extinfo contains the actual size of the data payload in the toast + * records. */ if (VARATT_IS_SHORT(dval)) { data_p = VARDATA_SHORT(dval); data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT; toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */ - toast_pointer.va_extsize = data_todo; + toast_pointer.va_extinfo = data_todo; /* no flags */ } else if (VARATT_IS_COMPRESSED(dval)) { @@ -1530,7 +1617,10 @@ toast_save_datum(Relation rel, Datum value, data_todo = VARSIZE(dval) - VARHDRSZ; /* rawsize in a compressed datum is just the size of the payload */ toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ; - toast_pointer.va_extsize = data_todo; + toast_pointer.va_extinfo = data_todo; + if (VARATT_IS_CUSTOM_COMPRESSED(dval)) + VARATT_EXTERNAL_SET_CUSTOM(toast_pointer); + /* Assert that the numbers look like it's compressed */ Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)); } @@ -1539,7 +1629,7 @@ toast_save_datum(Relation rel, Datum value, data_p = VARDATA(dval); data_todo = VARSIZE(dval) - VARHDRSZ; toast_pointer.va_rawsize = VARSIZE(dval); - toast_pointer.va_extsize = data_todo; + toast_pointer.va_extinfo = data_todo; /* no flags */ } /* @@ -1899,7 +1989,7 @@ toast_fetch_datum(struct varlena *attr) /* Must copy to access aligned fields */ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr); - ressize = toast_pointer.va_extsize; + ressize = VARATT_EXTERNAL_GET_SIZE(toast_pointer); numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1; result = (struct varlena *) palloc(ressize + VARHDRSZ); @@ -2084,7 +2174,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length) */ Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)); - attrsize = toast_pointer.va_extsize; + attrsize = VARATT_EXTERNAL_GET_SIZE(toast_pointer); totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1; if (sliceoffset >= attrsize) @@ -2280,15 +2370,26 @@ toast_decompress_datum(struct varlena *attr) Assert(VARATT_IS_COMPRESSED(attr)); - result = (struct varlena *) - palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); - SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); + if (VARATT_IS_CUSTOM_COMPRESSED(attr)) + { + CompressionOptions *cmoptions; + toast_compress_header_custom *hdr; - if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr), - VARSIZE(attr) - TOAST_COMPRESS_HDRSZ, - VARDATA(result), - TOAST_COMPRESS_RAWSIZE(attr)) < 0) - elog(ERROR, "compressed data is corrupted"); + hdr = (toast_compress_header_custom *) attr; + cmoptions = get_cached_compression_options(hdr->cmoptoid); + result = cmoptions->routine->decompress(cmoptions, attr); + } + else + { + result = (struct varlena *) + palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); + SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); + if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr), + VARSIZE(attr) - TOAST_COMPRESS_HDRSZ, + VARDATA(result), + TOAST_COMPRESS_RAWSIZE(attr)) < 0) + elog(ERROR, "compressed data is corrupted"); + } return result; } @@ -2390,3 +2491,65 @@ init_toast_snapshot(Snapshot toast_snapshot) InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken); } + +/* ---------- + * init_compression_options_cache + * + * Initialize a local cache for compression options. + */ +static void +init_compression_options_cache(void) +{ + HASHCTL ctl; + + compression_options_mcxt = AllocSetContextCreate(TopMemoryContext, + "compression options cache context", + ALLOCSET_DEFAULT_SIZES); + MemSet(&ctl, 0, sizeof(ctl)); + ctl.keysize = sizeof(Oid); + ctl.entrysize = sizeof(CompressionOptions); + ctl.hcxt = compression_options_mcxt; + compression_options_cache = hash_create("compression options cache", 100, &ctl, + HASH_ELEM | HASH_BLOBS | HASH_CONTEXT); +} + +/* ---------- + * get_cached_compression_options + * + * Get cached compression options structure or create it if it's not in cache. + * Cache is required because we can't afford for each tuple create + * CompressionMethodRoutine and parse its options. + */ +static CompressionOptions * +get_cached_compression_options(Oid cmoptoid) +{ + bool found; + CompressionOptions *result; + + Assert(OidIsValid(cmoptoid)); + if (!compression_options_cache) + init_compression_options_cache(); + + result = hash_search(compression_options_cache, &cmoptoid, HASH_ENTER, &found); + if (!found) + { + MemoryContext oldcxt; + + Assert(compression_options_mcxt); + oldcxt = MemoryContextSwitchTo(compression_options_mcxt); + result->cmoptoid = cmoptoid; + PG_TRY(); + { + result->routine = GetCompressionRoutine(cmoptoid); + result->options = GetCompressionOptionsList(cmoptoid); + } + PG_CATCH(); + { + hash_search(compression_options_cache, &cmoptoid, HASH_REMOVE, &found); + PG_RE_THROW(); + } + PG_END_TRY(); + MemoryContextSwitchTo(oldcxt); + } + return result; +} diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 8287de97a2..be6b460aee 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/backend/bootstrap/bootstrap.c @@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness) attrtypes[attnum]->attcacheoff = -1; attrtypes[attnum]->atttypmod = -1; attrtypes[attnum]->attislocal = true; + attrtypes[attnum]->attcompression = InvalidOid; if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL) { diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile index fd33426bad..c7cea974b1 100644 --- a/src/backend/catalog/Makefile +++ b/src/backend/catalog/Makefile @@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\ pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \ pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \ pg_subscription_rel.h toasting.h indexing.h \ - toasting.h indexing.h \ + pg_compression.h pg_compression_opt.h \ ) # location of Catalog.pm diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index ccde66a7dd..fd733a34a0 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] = gettext_noop("permission denied for publication %s"), /* ACL_KIND_SUBSCRIPTION */ gettext_noop("permission denied for subscription %s"), + /* ACL_KIND_COMPRESSION_METHOD */ + gettext_noop("permission denied for compression method %s"), }; static const char *const not_owner_msg[MAX_ACL_KIND] = diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c index 033c4358ea..e1bfc7c6bf 100644 --- a/src/backend/catalog/dependency.c +++ b/src/backend/catalog/dependency.c @@ -28,6 +28,8 @@ #include "catalog/pg_cast.h" #include "catalog/pg_collation.h" #include "catalog/pg_collation_fn.h" +#include "catalog/pg_compression.h" +#include "catalog/pg_compression_opt.h" #include "catalog/pg_constraint.h" #include "catalog/pg_constraint_fn.h" #include "catalog/pg_conversion.h" @@ -173,7 +175,9 @@ static const Oid object_classes[] = { PublicationRelationId, /* OCLASS_PUBLICATION */ PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */ SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */ - TransformRelationId /* OCLASS_TRANSFORM */ + TransformRelationId, /* OCLASS_TRANSFORM */ + CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */ + CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */ }; @@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags) DropTransformById(object->objectId); break; + case OCLASS_COMPRESSION_METHOD: + RemoveCompressionMethodById(object->objectId); + break; + + case OCLASS_COMPRESSION_OPTIONS: + RemoveCompressionOptionsById(object->objectId); + break; + /* * These global object types are not supported here. */ @@ -2512,6 +2524,12 @@ getObjectClass(const ObjectAddress *object) case TransformRelationId: return OCLASS_TRANSFORM; + + case CompressionMethodRelationId: + return OCLASS_COMPRESSION_METHOD; + + case CompressionOptRelationId: + return OCLASS_COMPRESSION_OPTIONS; } /* shouldn't get here */ diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl index 256a9c9c93..c5838fa779 100644 --- a/src/backend/catalog/genbki.pl +++ b/src/backend/catalog/genbki.pl @@ -451,6 +451,7 @@ sub emit_pgattr_row attisdropped => 'f', attislocal => 't', attinhcount => '0', + attcompression=> '0', attacl => '_null_', attoptions => '_null_', attfdwoptions => '_null_'); diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index 9e14880b99..4f958be51f 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -29,8 +29,10 @@ */ #include "postgres.h" +#include "access/compression.h" #include "access/htup_details.h" #include "access/multixact.h" +#include "access/reloptions.h" #include "access/sysattr.h" #include "access/transam.h" #include "access/xact.h" @@ -44,6 +46,8 @@ #include "catalog/partition.h" #include "catalog/pg_attrdef.h" #include "catalog/pg_collation.h" +#include "catalog/pg_compression.h" +#include "catalog/pg_compression_opt.h" #include "catalog/pg_constraint.h" #include "catalog/pg_constraint_fn.h" #include "catalog/pg_foreign_table.h" @@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel, values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal); values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount); values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation); + values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression); /* start out with empty permissions and empty options */ nulls[Anum_pg_attribute_attacl - 1] = true; @@ -707,6 +712,13 @@ AddNewAttributeTuples(Oid new_rel_oid, referenced.objectSubId = 0; recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); } + + if (OidIsValid(attr->attcompression)) + { + ObjectAddressSet(referenced, CompressionOptRelationId, + attr->attcompression); + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + } } /* @@ -1453,6 +1465,24 @@ DeleteRelationTuple(Oid relid) heap_close(pg_class_desc, RowExclusiveLock); } +/* + * CallCompressionDropCallback + * + * Call drop callback from compression routine. + */ +static void +CallCompressionDropCallback(Form_pg_attribute att) +{ + CompressionMethodRoutine *cmr = GetCompressionRoutine(att->attcompression); + + if (cmr->drop) + { + List *options = GetCompressionOptionsList(att->attcompression); + + cmr->drop(att, options); + } +} + /* * DeleteAttributeTuples * @@ -1483,7 +1513,14 @@ DeleteAttributeTuples(Oid relid) /* Delete all the matching tuples */ while ((atttup = systable_getnext(scan)) != NULL) + { + Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup); + + if (OidIsValid(att->attcompression)) + CallCompressionDropCallback(att); + CatalogTupleDelete(attrel, &atttup->t_self); + } /* Clean up after the scan */ systable_endscan(scan); @@ -1576,6 +1613,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum) else { /* Dropping user attributes is lots harder */ + if (OidIsValid(attStruct->attcompression)) + CallCompressionDropCallback(attStruct); /* Mark the attribute as dropped */ attStruct->attisdropped = true; @@ -1597,6 +1636,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum) /* We don't want to keep stats for it anymore */ attStruct->attstattarget = 0; + attStruct->attcompression = InvalidOid; + /* * Change the column name to something that isn't likely to conflict */ diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 0125c18bc1..15942564aa 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation, to->atttypmod = exprTypmod(indexkey); to->attislocal = true; to->attcollation = collationObjectId[i]; + to->attcompression = InvalidOid; ReleaseSysCache(tuple); @@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation, to->attbyval = typeTup->typbyval; to->attalign = typeTup->typalign; to->attstorage = typeTup->typstorage; + to->attcompression = InvalidOid; ReleaseSysCache(tuple); } diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c index 8d55c76fc4..9fd1cb763d 100644 --- a/src/backend/catalog/objectaddress.c +++ b/src/backend/catalog/objectaddress.c @@ -29,6 +29,8 @@ #include "catalog/pg_default_acl.h" #include "catalog/pg_event_trigger.h" #include "catalog/pg_collation.h" +#include "catalog/pg_compression.h" +#include "catalog/pg_compression_opt.h" #include "catalog/pg_constraint.h" #include "catalog/pg_constraint_fn.h" #include "catalog/pg_conversion.h" @@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] = InvalidAttrNumber, /* no ACL (same as relation) */ ACL_KIND_STATISTICS, true + }, + { + CompressionMethodRelationId, + CompressionMethodOidIndexId, + COMPRESSIONMETHODOID, + COMPRESSIONMETHODNAME, + Anum_pg_compression_cmname, + InvalidAttrNumber, + InvalidAttrNumber, + InvalidAttrNumber, + -1, + true + }, + { + CompressionOptRelationId, + CompressionOptionsOidIndexId, + COMPRESSIONOPTIONSOID, + -1, + InvalidAttrNumber, + InvalidAttrNumber, + InvalidAttrNumber, + InvalidAttrNumber, + -1, + false, } }; @@ -712,6 +738,10 @@ static const struct object_type_map /* OBJECT_STATISTIC_EXT */ { "statistics object", OBJECT_STATISTIC_EXT + }, + /* OCLASS_COMPRESSION_METHOD */ + { + "compression method", OBJECT_COMPRESSION_METHOD } }; @@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object, case OBJECT_ACCESS_METHOD: case OBJECT_PUBLICATION: case OBJECT_SUBSCRIPTION: + case OBJECT_COMPRESSION_METHOD: address = get_object_address_unqualified(objtype, (Value *) object, missing_ok); break; @@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype, address.objectId = get_subscription_oid(name, missing_ok); address.objectSubId = 0; break; + case OBJECT_COMPRESSION_METHOD: + address.classId = CompressionMethodRelationId; + address.objectId = get_compression_method_oid(name, missing_ok); + address.objectSubId = 0; + break; default: elog(ERROR, "unrecognized objtype: %d", (int) objtype); /* placate compiler, which doesn't know elog won't return */ @@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS) case OBJECT_SCHEMA: case OBJECT_SUBSCRIPTION: case OBJECT_TABLESPACE: + case OBJECT_COMPRESSION_METHOD: if (list_length(name) != 1) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), @@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address, case OBJECT_TSPARSER: case OBJECT_TSTEMPLATE: case OBJECT_ACCESS_METHOD: + case OBJECT_COMPRESSION_METHOD: /* We treat these object types as being owned by superusers */ if (!superuser_arg(roleid)) ereport(ERROR, (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), errmsg("must be superuser"))); break; + case OBJECT_STATISTIC_EXT: if (!pg_statistics_object_ownercheck(address.objectId, roleid)) aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId); @@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object) break; } + case OCLASS_COMPRESSION_METHOD: + { + char *name = get_compression_method_name(object->objectId); + + if (!name) + elog(ERROR, "cache lookup failed for compression method %u", + object->objectId); + appendStringInfo(&buffer, _("compression method %s"), name); + pfree(name); + break; + } + + case OCLASS_COMPRESSION_OPTIONS: + { + char *name = get_compression_method_name_for_opt(object->objectId); + + appendStringInfo(&buffer, _("compression options for %s"), name); + pfree(name); + break; + } + case OCLASS_TRANSFORM: { HeapTuple trfTup; @@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object) appendStringInfoString(&buffer, "transform"); break; + case OCLASS_COMPRESSION_METHOD: + appendStringInfoString(&buffer, "compression method"); + break; + + case OCLASS_COMPRESSION_OPTIONS: + appendStringInfoString(&buffer, "compression options"); + break; + /* * There's intentionally no default: case here; we want the * compiler to warn if a new OCLASS hasn't been handled above. @@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object, break; } + case OCLASS_COMPRESSION_METHOD: + { + char *cmname = get_compression_method_name(object->objectId); + + if (!cmname) + elog(ERROR, "cache lookup failed for compression method %u", + object->objectId); + appendStringInfoString(&buffer, quote_identifier(cmname)); + if (objname) + *objname = list_make1(cmname); + else + pfree(cmname); + break; + } + + case OCLASS_COMPRESSION_OPTIONS: + { + appendStringInfo(&buffer, "%u", + object->objectId); + if (objname) + *objname = list_make1(psprintf("%u", object->objectId)); + break; + } + case OCLASS_CONSTRAINT: { HeapTuple conTup; diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c index e02d312008..531a820464 100644 --- a/src/backend/catalog/pg_type.c +++ b/src/backend/catalog/pg_type.c @@ -22,6 +22,7 @@ #include "catalog/indexing.h" #include "catalog/objectaccess.h" #include "catalog/pg_collation.h" +#include "catalog/pg_compression.h" #include "catalog/pg_namespace.h" #include "catalog/pg_proc.h" #include "catalog/pg_type.h" diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile index 4a6c99e090..e23abf64f1 100644 --- a/src/backend/commands/Makefile +++ b/src/backend/commands/Makefile @@ -13,8 +13,8 @@ top_builddir = ../../.. include $(top_builddir)/src/Makefile.global OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \ - collationcmds.o constraint.o conversioncmds.o copy.o createas.o \ - dbcommands.o define.o discard.o dropcmds.o \ + collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \ + createas.o dbcommands.o define.o discard.o dropcmds.o \ event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \ indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \ policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \ diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c index 4f8147907c..4f18e4083f 100644 --- a/src/backend/commands/alter.c +++ b/src/backend/commands/alter.c @@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt) case OBJECT_TSTEMPLATE: case OBJECT_PUBLICATION: case OBJECT_SUBSCRIPTION: + case OBJECT_COMPRESSION_METHOD: { ObjectAddress address; Relation catalog; @@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt, case OBJECT_TSDICTIONARY: case OBJECT_TSPARSER: case OBJECT_TSTEMPLATE: + case OBJECT_COMPRESSION_METHOD: { Relation catalog; Relation relation; @@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid, case OCLASS_PUBLICATION_REL: case OCLASS_SUBSCRIPTION: case OCLASS_TRANSFORM: + case OCLASS_COMPRESSION_METHOD: + case OCLASS_COMPRESSION_OPTIONS: /* ignore object types that don't have schema-qualified names */ break; diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c new file mode 100644 index 0000000000..fa2ad8a864 --- /dev/null +++ b/src/backend/commands/compressioncmds.c @@ -0,0 +1,489 @@ +/*------------------------------------------------------------------------- + * + * compressioncmds.c + * Routines for SQL commands that manipulate compression methods. + * + * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/commands/compressioncmds.c + *------------------------------------------------------------------------- + */ +#include "postgres.h" +#include "miscadmin.h" + +#include "access/compression.h" +#include "access/heapam.h" +#include "access/htup_details.h" +#include "access/reloptions.h" +#include "catalog/catalog.h" +#include "catalog/dependency.h" +#include "catalog/indexing.h" +#include "catalog/pg_compression.h" +#include "catalog/pg_compression_opt.h" +#include "catalog/pg_proc.h" +#include "catalog/pg_type.h" +#include "commands/defrem.h" +#include "parser/parse_func.h" +#include "utils/builtins.h" +#include "utils/lsyscache.h" +#include "utils/rel.h" +#include "utils/syscache.h" + +static CompressionMethodRoutine * get_compression_method_routine(Oid cmhandler, Oid typeid); + +/* + * Convert a handler function name to an Oid. If the return type of the + * function doesn't match the given AM type, an error is raised. + * + * This function either return valid function Oid or throw an error. + */ +static Oid +LookupCompressionHandlerFunc(List *handlerName) +{ + static const Oid funcargtypes[1] = {INTERNALOID}; + Oid handlerOid; + + if (handlerName == NIL) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("handler function is not specified"))); + + /* handlers have one argument of type internal */ + handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false); + + /* check that handler has the correct return type */ + if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID) + ereport(ERROR, + (errcode(ERRCODE_WRONG_OBJECT_TYPE), + errmsg("function %s must return type %s", + NameListToString(handlerName), + "compression_handler"))); + + return handlerOid; +} + +static ObjectAddress +CreateCompressionMethod(char *cmName, List *handlerName) +{ + Relation rel; + ObjectAddress myself; + ObjectAddress referenced; + Oid cmoid; + Oid cmhandler; + bool nulls[Natts_pg_compression]; + Datum values[Natts_pg_compression]; + HeapTuple tup; + + rel = heap_open(CompressionMethodRelationId, RowExclusiveLock); + + /* Check if name is used */ + cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName)); + if (OidIsValid(cmoid)) + ereport(ERROR, + (errcode(ERRCODE_DUPLICATE_OBJECT), + errmsg("compression method \"%s\" already exists", + cmName))); + + /* + * Get the handler function oid and compression method routine + */ + cmhandler = LookupCompressionHandlerFunc(handlerName); + + /* + * Insert tuple into pg_compression. + */ + memset(values, 0, sizeof(values)); + memset(nulls, false, sizeof(nulls)); + + values[Anum_pg_compression_cmname - 1] = + DirectFunctionCall1(namein, CStringGetDatum(cmName)); + values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler); + + tup = heap_form_tuple(RelationGetDescr(rel), values, nulls); + + cmoid = CatalogTupleInsert(rel, tup); + heap_freetuple(tup); + + ObjectAddressSet(myself, CompressionMethodRelationId, cmoid); + + /* Record dependency on handler function */ + ObjectAddressSet(referenced, ProcedureRelationId, cmhandler); + + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + recordDependencyOnCurrentExtension(&myself, false); + + heap_close(rel, RowExclusiveLock); + + return myself; +} + +/* + * CREATE COMPRESSION METHOD .. HANDLER .. + */ +ObjectAddress +DefineCompressionMethod(List *names, List *parameters) +{ + char *cmName; + ListCell *pl; + DefElem *handlerEl = NULL; + + if (list_length(names) != 1) + elog(ERROR, "compression method name cannot be qualified"); + + cmName = strVal(linitial(names)); + + /* Must be super user */ + if (!superuser()) + ereport(ERROR, + (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), + errmsg("permission denied to create compression method \"%s\"", + cmName), + errhint("Must be superuser to create an compression method."))); + + foreach(pl, parameters) + { + DefElem *defel = (DefElem *) lfirst(pl); + DefElem **defelp; + + if (pg_strcasecmp(defel->defname, "handler") == 0) + defelp = &handlerEl; + else + { + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("compression method attribute \"%s\" not recognized", + defel->defname))); + break; + } + + *defelp = defel; + } + + if (!handlerEl) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("compression method handler is not specified"))); + + return CreateCompressionMethod(cmName, (List *) handlerEl->arg); +} + +void +RemoveCompressionMethodById(Oid cmOid) +{ + Relation relation; + HeapTuple tup; + + if (!superuser()) + ereport(ERROR, + (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), + errmsg("must be superuser to drop compression methods"))); + + relation = heap_open(CompressionMethodRelationId, RowExclusiveLock); + + tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid)); + if (!HeapTupleIsValid(tup)) + elog(ERROR, "cache lookup failed for compression method %u", cmOid); + + CatalogTupleDelete(relation, &tup->t_self); + + ReleaseSysCache(tup); + + heap_close(relation, RowExclusiveLock); +} + +/* + * Create new record in pg_compression_opt + */ +Oid +CreateCompressionOptions(Form_pg_attribute attr, Oid cmid, List *options) +{ + Relation rel; + HeapTuple tup, + newtup; + Oid cmoptoid; + Datum values[Natts_pg_compression_opt]; + bool nulls[Natts_pg_compression_opt]; + + ObjectAddress myself, + ref1, + ref2, + ref3; + + CompressionMethodRoutine *routine; + Form_pg_compression cmform; + + /* Initialize buffers for new tuple values */ + memset(values, 0, sizeof(values)); + memset(nulls, false, sizeof(nulls)); + + /* Get handler function OID for the compression method */ + tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid)); + if (!HeapTupleIsValid(tup)) + elog(ERROR, "cache lookup failed for compression method %u", cmid); + cmform = (Form_pg_compression) GETSTRUCT(tup); + routine = get_compression_method_routine(cmform->cmhandler, attr->atttypid); + + if (routine->configure == NULL && options != NIL) + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("the compression method \"%s\" does not take any options", + NameStr(cmform->cmname)))); + else if (routine->configure && options != NIL) + routine->configure(attr, options); + + rel = heap_open(CompressionOptRelationId, RowExclusiveLock); + + cmoptoid = GetNewOidWithIndex(rel, CompressionOptionsOidIndexId, + Anum_pg_compression_opt_cmoptoid); + values[Anum_pg_compression_opt_cmoptoid - 1] = ObjectIdGetDatum(cmoptoid); + values[Anum_pg_compression_opt_cmname - 1] = NameGetDatum(&cmform->cmname); + values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmform->cmhandler); + + if (options) + values[Anum_pg_compression_opt_cmoptions - 1] = optionListToArray(options); + else + nulls[Anum_pg_compression_opt_cmoptions - 1] = true; + + newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls); + CatalogTupleInsert(rel, newtup); + heap_freetuple(newtup); + + ReleaseSysCache(tup); + + ObjectAddressSet(myself, CompressionOptRelationId, cmoptoid); + ObjectAddressSet(ref1, ProcedureRelationId, cmform->cmhandler); + ObjectAddressSubSet(ref2, RelationRelationId, attr->attrelid, attr->attnum); + ObjectAddressSet(ref3, CompressionMethodRelationId, cmid); + + recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL); + recordDependencyOn(&ref2, &myself, DEPENDENCY_NORMAL); + recordDependencyOn(&myself, &ref3, DEPENDENCY_NORMAL); + recordDependencyOnCurrentExtension(&myself, false); + heap_close(rel, RowExclusiveLock); + + return cmoptoid; +} + +/* + * Remove the compression options record from pg_compression_opt + */ +void +RemoveCompressionOptionsById(Oid cmoptoid) +{ + Relation relation; + HeapTuple tup; + + if (!superuser()) + ereport(ERROR, + (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), + errmsg("must be superuser to drop compression options"))); + + tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid)); + if (!HeapTupleIsValid(tup)) + elog(ERROR, "cache lookup failed for compression options %u", cmoptoid); + + relation = heap_open(CompressionOptRelationId, RowExclusiveLock); + CatalogTupleDelete(relation, &tup->t_self); + heap_close(relation, RowExclusiveLock); + ReleaseSysCache(tup); +} + +ColumnCompression * +GetColumnCompressionForAttribute(Form_pg_attribute att) +{ + HeapTuple tuple; + Form_pg_compression_opt cmoptform; + ColumnCompression *compression = makeNode(ColumnCompression); + + /* Get handler function OID for the compression method */ + tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, + ObjectIdGetDatum(att->attcompression)); + + if (!HeapTupleIsValid(tuple)) + elog(ERROR, "cache lookup failed for compression options %u", + att->attcompression); + + cmoptform = (Form_pg_compression_opt) GETSTRUCT(tuple); + compression->methodName = pstrdup(NameStr(cmoptform->cmname)); + compression->options = GetCompressionOptionsList(att->attcompression); + ReleaseSysCache(tuple); + + return compression; +} + +void +CheckCompressionMismatch(ColumnCompression * c1, ColumnCompression * c2, + const char *attributeName) +{ + if (strcmp(c1->methodName, c2->methodName)) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("column \"%s\" has a compression method conflict", + attributeName), + errdetail("%s versus %s", c1->methodName, c2->methodName))); + + if (!equal(c1->options, c2->options)) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("column \"%s\" has a compression options conflict", + attributeName), + errdetail("(%s) versus (%s)", + formatRelOptions(c1->options), + formatRelOptions(c2->options)))); +} + +/* + * get_compression_method_oid + * + * If missing_ok is false, throw an error if compression method not found. + * If missing_ok is true, just return InvalidOid. + */ +Oid +get_compression_method_oid(const char *cmname, bool missing_ok) +{ + HeapTuple tup; + Oid oid = InvalidOid; + + tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname)); + if (HeapTupleIsValid(tup)) + { + oid = HeapTupleGetOid(tup); + ReleaseSysCache(tup); + } + + if (!OidIsValid(oid) && !missing_ok) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("compression method \"%s\" does not exist", cmname))); + + return oid; +} + +/* + * get_compression_method_name + * + * given an compression method OID, look up its name. + */ +char * +get_compression_method_name(Oid cmOid) +{ + HeapTuple tup; + char *result = NULL; + + tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid)); + if (HeapTupleIsValid(tup)) + { + Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup); + + result = pstrdup(NameStr(cmform->cmname)); + ReleaseSysCache(tup); + } + return result; +} + +/* + * get_compression_method_name_for_opt + * + * given an compression options OID, look up a name for used compression method. + */ +char * +get_compression_method_name_for_opt(Oid cmoptoid) +{ + HeapTuple tup; + char *result = NULL; + Form_pg_compression_opt cmoptform; + + tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid)); + + if (!HeapTupleIsValid(tup)) + elog(ERROR, "cache lookup failed for compression options %u", cmoptoid); + + cmoptform = (Form_pg_compression_opt) GETSTRUCT(tup); + result = pstrdup(NameStr(cmoptform->cmname)); + ReleaseSysCache(tup); + + return result; +} + +/* get_compression_options */ +List * +GetCompressionOptionsList(Oid cmoptoid) +{ + HeapTuple tuple; + List *result = NULL; + bool isnull; + Datum cmoptions; + + /* Get handler function OID for the compression method */ + tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid)); + + if (!HeapTupleIsValid(tuple)) + elog(ERROR, "cache lookup failed for compression options %u", cmoptoid); + + cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple, + Anum_pg_compression_opt_cmoptions, &isnull); + + if (!isnull) + result = untransformRelOptions(cmoptions); + + ReleaseSysCache(tuple); + return result; +} + +CompressionMethodRoutine * +GetCompressionRoutine(Oid cmoptoid) +{ + HeapTuple tuple; + Form_pg_compression_opt cmopt; + regproc cmhandler; + + /* Get handler function OID for the compression method */ + tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid)); + + if (!HeapTupleIsValid(tuple)) + elog(ERROR, "cache lookup failed for compression options %u", cmoptoid); + + cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple); + cmhandler = cmopt->cmhandler; + + /* Complain if handler OID is invalid */ + if (!RegProcedureIsValid(cmhandler)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("could not find compression method handler for compression options %u", + cmoptoid))); + + ReleaseSysCache(tuple); + + /* And finally, call the handler function to get the API struct. */ + return get_compression_method_routine(cmhandler, InvalidOid); +} + +/* + * Call the specified compression method handler + * routine to get its CompressionMethodRoutine struct, + * which will be palloc'd in the caller's context. + */ +static CompressionMethodRoutine * +get_compression_method_routine(Oid cmhandler, Oid typeid) +{ + Datum datum; + CompressionMethodRoutine *routine; + CompressionMethodOpArgs opargs; + + opargs.typeid = typeid; + opargs.cmhanderid = cmhandler; + + datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs)); + routine = (CompressionMethodRoutine *) DatumGetPointer(datum); + + if (routine == NULL || !IsA(routine, CompressionMethodRoutine)) + elog(ERROR, "compression method handler function %u " + "did not return an CompressionMethodRoutine struct", + cmhandler); + + return routine; +} diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c index 13eb9e34ba..f84588ed31 100644 --- a/src/backend/commands/copy.c +++ b/src/backend/commands/copy.c @@ -2781,8 +2781,8 @@ CopyFrom(CopyState cstate) List *recheckIndexes = NIL; /* OK, store the tuple and create index entries for it */ - heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid, - hi_options, bistate); + heap_insert(resultRelInfo->ri_RelationDesc, tuple, + mycid, hi_options, bistate); if (resultRelInfo->ri_NumIndices > 0) recheckIndexes = ExecInsertIndexTuples(slot, diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index 4d77411a68..aba2087839 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -112,7 +112,8 @@ create_ctas_internal(List *attrList, IntoClause *into) * Create the relation. (This will error out if there's an existing view, * so we don't need more code to complain if "replace" is false.) */ - intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL); + intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL, + NULL); /* * If necessary, create a TOAST table for the target table. Note that diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c index 2b30677d6f..8611db22ec 100644 --- a/src/backend/commands/dropcmds.c +++ b/src/backend/commands/dropcmds.c @@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object) name = NameListToString(castNode(List, object)); } break; + case OBJECT_COMPRESSION_METHOD: + msg = gettext_noop("compression method \"%s\" does not exist, skipping"); + name = strVal((Value *) object); + break; case OBJECT_CONVERSION: if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name)) { diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c index fa7d0d015a..a2eee78a64 100644 --- a/src/backend/commands/event_trigger.c +++ b/src/backend/commands/event_trigger.c @@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = { {"CAST", true}, {"CONSTRAINT", true}, {"COLLATION", true}, + {"COMPRESSION METHOD", true}, {"CONVERSION", true}, {"DATABASE", false}, {"DOMAIN", true}, @@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype) case OBJECT_CAST: case OBJECT_COLUMN: case OBJECT_COLLATION: + case OBJECT_COMPRESSION_METHOD: case OBJECT_CONVERSION: case OBJECT_DEFACL: case OBJECT_DEFAULT: @@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass) case OCLASS_DATABASE: case OCLASS_TBLSPACE: case OCLASS_ROLE: + case OCLASS_COMPRESSION_OPTIONS: /* no support for global objects */ return false; case OCLASS_EVENT_TRIGGER: @@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass) case OCLASS_PUBLICATION_REL: case OCLASS_SUBSCRIPTION: case OCLASS_TRANSFORM: + case OCLASS_COMPRESSION_METHOD: return true; /* diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c index 9ad991507f..b840309d03 100644 --- a/src/backend/commands/foreigncmds.c +++ b/src/backend/commands/foreigncmds.c @@ -61,7 +61,7 @@ static void import_error_callback(void *arg); * processing, hence any validation should be done before this * conversion. */ -static Datum +Datum optionListToArray(List *options) { ArrayBuildState *astate = NULL; diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c index 5e1b0fe289..eb9f72dcba 100644 --- a/src/backend/commands/sequence.c +++ b/src/backend/commands/sequence.c @@ -212,7 +212,8 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq) stmt->tablespacename = NULL; stmt->if_not_exists = seq->if_not_exists; - address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL); + address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL, + NULL); seqoid = address.objectId; Assert(seqoid != InvalidOid); diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d19846d005..f31be71e65 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -14,6 +14,7 @@ */ #include "postgres.h" +#include "access/compression.h" #include "access/genam.h" #include "access/heapam.h" #include "access/multixact.h" @@ -33,6 +34,8 @@ #include "catalog/partition.h" #include "catalog/pg_am.h" #include "catalog/pg_collation.h" +#include "catalog/pg_compression.h" +#include "catalog/pg_compression_opt.h" #include "catalog/pg_constraint.h" #include "catalog/pg_constraint_fn.h" #include "catalog/pg_depend.h" @@ -41,6 +44,7 @@ #include "catalog/pg_inherits_fn.h" #include "catalog/pg_namespace.h" #include "catalog/pg_opclass.h" +#include "catalog/pg_proc.h" #include "catalog/pg_tablespace.h" #include "catalog/pg_trigger.h" #include "catalog/pg_type.h" @@ -90,6 +94,7 @@ #include "storage/smgr.h" #include "utils/acl.h" #include "utils/builtins.h" +#include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" @@ -459,6 +464,8 @@ static void ATExecGenericOptions(Relation rel, List *options); static void ATExecEnableRowSecurity(Relation rel); static void ATExecDisableRowSecurity(Relation rel); static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls); +static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel, + const char *column, ColumnCompression * compression, LOCKMODE lockmode); static void copy_relation_data(SMgrRelation rel, SMgrRelation dst, ForkNumber forkNum, char relpersistence); @@ -503,7 +510,8 @@ static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name); */ ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, - ObjectAddress *typaddress, const char *queryString) + ObjectAddress *typaddress, const char *queryString, + Node **pAlterStmt) { char relname[NAMEDATALEN]; Oid namespaceId; @@ -523,6 +531,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, static char *validnsps[] = HEAP_RELOPT_NAMESPACES; Oid ofTypeId; ObjectAddress address; + AlterTableStmt *alterStmt = NULL; /* * Truncate relname to appropriate length (probably a waste of time, as @@ -724,6 +733,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, if (colDef->identity) attr->attidentity = colDef->identity; + + transformColumnCompression(colDef, stmt->relation, &alterStmt); } /* @@ -921,6 +932,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, */ relation_close(rel, NoLock); + if (pAlterStmt) + *pAlterStmt = (Node *) alterStmt; + else + Assert(!alterStmt); + return address; } @@ -1610,6 +1626,7 @@ storage_name(char c) } } + /*---------- * MergeAttributes * Returns new schema given initial schema and superclasses. @@ -1944,6 +1961,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence, storage_name(def->storage), storage_name(attribute->attstorage)))); + if (OidIsValid(attribute->attcompression)) + { + ColumnCompression *compression = + GetColumnCompressionForAttribute(attribute); + + if (!def->compression) + def->compression = compression; + else + CheckCompressionMismatch(def->compression, + compression, + attributeName); + } + def->inhcount++; /* Merge of NOT NULL constraints = OR 'em together */ def->is_not_null |= attribute->attnotnull; @@ -1971,6 +2001,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence, def->collOid = attribute->attcollation; def->constraints = NIL; def->location = -1; + if (OidIsValid(attribute->attcompression)) + def->compression = + GetColumnCompressionForAttribute(attribute); inhSchema = lappend(inhSchema, def); newattno[parent_attno - 1] = ++child_attno; } @@ -2180,6 +2213,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence, storage_name(def->storage), storage_name(newdef->storage)))); + if (!def->compression) + def->compression = newdef->compression; + else if (newdef->compression) + CheckCompressionMismatch(def->compression, + newdef->compression, + attributeName); + /* Mark the column as locally defined */ def->is_local = true; /* Merge of NOT NULL constraints = OR 'em together */ @@ -3279,6 +3319,7 @@ AlterTableGetLockLevel(List *cmds) */ case AT_GenericOptions: case AT_AlterColumnGenericOptions: + case AT_AlterColumnCompression: cmd_lockmode = AccessExclusiveLock; break; @@ -3770,6 +3811,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AlterColumnCompression: + ATSimplePermissions(rel, ATT_TABLE); + /* FIXME This command never recurses */ + /* No command-specific prep needed */ + pass = AT_PASS_MISC; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -4118,6 +4165,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel, case AT_DetachPartition: ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name); break; + case AT_AlterColumnCompression: + ATExecAlterColumnCompression(tab, rel, cmd->name, + (ColumnCompression *) cmd->def, + lockmode); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5338,6 +5390,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel, attribute.attislocal = colDef->is_local; attribute.attinhcount = colDef->inhcount; attribute.attcollation = collOid; + attribute.attcompression = InvalidOid; + /* attribute.attacl is handled by InsertPgAttributeTuple */ ReleaseSysCache(typeTuple); @@ -6438,6 +6492,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc errmsg("cannot alter system column \"%s\"", colName))); + if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm') + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("compressed columns can only have storage MAIN or EXTENDED"))); + /* * safety check: do not allow toasted storage modes unless column datatype * is TOAST-aware. @@ -9361,6 +9420,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel, case OCLASS_PUBLICATION_REL: case OCLASS_SUBSCRIPTION: case OCLASS_TRANSFORM: + case OCLASS_COMPRESSION_METHOD: + case OCLASS_COMPRESSION_OPTIONS: /* * We don't expect any of these sorts of objects to depend on @@ -9410,7 +9471,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel, if (!(foundDep->refclassid == TypeRelationId && foundDep->refobjid == attTup->atttypid) && !(foundDep->refclassid == CollationRelationId && - foundDep->refobjid == attTup->attcollation)) + foundDep->refobjid == attTup->attcollation) && + !(foundDep->refclassid == CompressionMethodRelationId && + foundDep->refobjid == attTup->attcompression)) elog(ERROR, "found unexpected dependency for column"); CatalogTupleDelete(depRel, &depTup->t_self); @@ -9432,6 +9495,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel, attTup->attbyval = tform->typbyval; attTup->attalign = tform->typalign; attTup->attstorage = tform->typstorage; + attTup->attcompression = InvalidOid; ReleaseSysCache(typeTuple); @@ -12484,6 +12548,83 @@ ATExecGenericOptions(Relation rel, List *options) heap_freetuple(tuple); } +static void +ATExecAlterColumnCompression(AlteredTableInfo *tab, + Relation rel, + const char *column, + ColumnCompression * compression, + LOCKMODE lockmode) +{ + Relation attrel; + HeapTuple atttuple; + Form_pg_attribute atttableform; + AttrNumber attnum; + HeapTuple newtuple; + Datum values[Natts_pg_attribute]; + bool nulls[Natts_pg_attribute]; + bool replace[Natts_pg_attribute]; + + attrel = heap_open(AttributeRelationId, RowExclusiveLock); + + atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column); + if (!HeapTupleIsValid(atttuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + column, RelationGetRelationName(rel)))); + + /* Prevent them from altering a system attribute */ + atttableform = (Form_pg_attribute) GETSTRUCT(atttuple); + attnum = atttableform->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", column))); + + if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm') + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("storage for \"%s\" should be MAIN or EXTENDED", column))); + + /* Initialize buffers for new tuple values */ + memset(values, 0, sizeof(values)); + memset(nulls, false, sizeof(nulls)); + memset(replace, false, sizeof(replace)); + + if (compression->methodName) + { + /* SET COMPRESSED */ + Oid cmid, + cmoptoid; + + cmid = get_compression_method_oid(compression->methodName, false); + cmoptoid = CreateCompressionOptions(atttableform, cmid, compression->options); + + values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid); + replace[Anum_pg_attribute_attcompression - 1] = true; + + } + else + { + /* SET NOT COMPRESSED */ + values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid); + replace[Anum_pg_attribute_attcompression - 1] = true; + } + + newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel), + values, nulls, replace); + CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple); + heap_freetuple(newtuple); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), + atttableform->attnum); + + ReleaseSysCache(atttuple); + heap_close(attrel, RowExclusiveLock); +} + + /* * Preparation phase for SET LOGGED/UNLOGGED * diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c index f86af4c054..be4a6c5f38 100644 --- a/src/backend/commands/typecmds.c +++ b/src/backend/commands/typecmds.c @@ -40,6 +40,7 @@ #include "catalog/pg_am.h" #include "catalog/pg_authid.h" #include "catalog/pg_collation.h" +#include "catalog/pg_compression.h" #include "catalog/pg_constraint.h" #include "catalog/pg_constraint_fn.h" #include "catalog/pg_depend.h" @@ -2182,7 +2183,7 @@ DefineCompositeType(RangeVar *typevar, List *coldeflist) * Finally create the relation. This also creates the type. */ DefineRelation(createStmt, RELKIND_COMPOSITE_TYPE, InvalidOid, &address, - NULL); + NULL, NULL); return address; } diff --git a/src/backend/commands/view.c b/src/backend/commands/view.c index c1e80e61d4..cc0182075e 100644 --- a/src/backend/commands/view.c +++ b/src/backend/commands/view.c @@ -251,7 +251,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace, * false). */ address = DefineRelation(createStmt, RELKIND_VIEW, InvalidOid, NULL, - NULL); + NULL, NULL); Assert(address.objectId != InvalidOid); /* Make the new view relation visible */ diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c index 5dfc49deb9..3a80e997a9 100644 --- a/src/backend/executor/execScan.c +++ b/src/backend/executor/execScan.c @@ -302,6 +302,9 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc var->vartypmod != -1)) return false; /* type mismatch */ + if (OidIsValid(att_tup->attcompression)) + return false; + tlist_item = lnext(tlist_item); } diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index e48921fff1..c9de2344fe 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -2806,6 +2806,7 @@ _copyColumnDef(const ColumnDef *from) COPY_STRING_FIELD(colname); COPY_NODE_FIELD(typeName); + COPY_NODE_FIELD(compression); COPY_SCALAR_FIELD(inhcount); COPY_SCALAR_FIELD(is_local); COPY_SCALAR_FIELD(is_not_null); @@ -2824,6 +2825,17 @@ _copyColumnDef(const ColumnDef *from) return newnode; } +static ColumnCompression * +_copyColumnCompression(const ColumnCompression * from) +{ + ColumnCompression *newnode = makeNode(ColumnCompression); + + COPY_STRING_FIELD(methodName); + COPY_NODE_FIELD(options); + + return newnode; +} + static Constraint * _copyConstraint(const Constraint *from) { @@ -5473,6 +5485,9 @@ copyObjectImpl(const void *from) case T_ColumnDef: retval = _copyColumnDef(from); break; + case T_ColumnCompression: + retval = _copyColumnCompression(from); + break; case T_Constraint: retval = _copyConstraint(from); break; diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index 2a83da9aa9..d618cc637e 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -2544,6 +2544,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b) { COMPARE_STRING_FIELD(colname); COMPARE_NODE_FIELD(typeName); + COMPARE_NODE_FIELD(compression); COMPARE_SCALAR_FIELD(inhcount); COMPARE_SCALAR_FIELD(is_local); COMPARE_SCALAR_FIELD(is_not_null); @@ -2562,6 +2563,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b) return true; } +static bool +_equalColumnCompression(const ColumnCompression * a, const ColumnCompression * b) +{ + COMPARE_STRING_FIELD(methodName); + COMPARE_NODE_FIELD(options); + + return true; +} + static bool _equalConstraint(const Constraint *a, const Constraint *b) { @@ -3615,6 +3625,9 @@ equal(const void *a, const void *b) case T_ColumnDef: retval = _equalColumnDef(a, b); break; + case T_ColumnCompression: + retval = _equalColumnCompression(a, b); + break; case T_Constraint: retval = _equalConstraint(a, b); break; diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c index c2a93b2d4c..81e334a1d3 100644 --- a/src/backend/nodes/nodeFuncs.c +++ b/src/backend/nodes/nodeFuncs.c @@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node, if (walker(coldef->typeName, context)) return true; + if (walker(coldef->compression, context)) + return true; if (walker(coldef->raw_default, context)) return true; if (walker(coldef->collClause, context)) @@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node, /* for now, constraints are ignored */ } break; + case T_ColumnCompression: + { + ColumnCompression *colcmp = (ColumnCompression *) node; + + if (walker(colcmp->options, context)) + return true; + } + break; case T_IndexElem: { IndexElem *indelem = (IndexElem *) node; diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index c97ee24ade..5ae1db146e 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -2799,6 +2799,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node) WRITE_STRING_FIELD(colname); WRITE_NODE_FIELD(typeName); + WRITE_NODE_FIELD(compression); WRITE_INT_FIELD(inhcount); WRITE_BOOL_FIELD(is_local); WRITE_BOOL_FIELD(is_not_null); @@ -2815,6 +2816,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node) WRITE_LOCATION_FIELD(location); } +static void +_outColumnCompression(StringInfo str, const ColumnCompression * node) +{ + WRITE_NODE_TYPE("COLUMNCOMPRESSION"); + + WRITE_STRING_FIELD(methodName); + WRITE_NODE_FIELD(options); +} + static void _outTypeName(StringInfo str, const TypeName *node) { @@ -4106,6 +4116,9 @@ outNode(StringInfo str, const void *obj) case T_ColumnDef: _outColumnDef(str, obj); break; + case T_ColumnCompression: + _outColumnCompression(str, obj); + break; case T_TypeName: _outTypeName(str, obj); break; diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c301ca465d..122d3de4d6 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -397,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); transform_element_list transform_type_list TriggerTransitions TriggerReferencing publication_name_list + optCompressionParameters vacuum_relation_list opt_vacuum_relation_list %type group_by_list @@ -582,6 +583,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %type hash_partbound partbound_datum_list range_datum_list %type hash_partbound_elem +%type columnCompression optColumnCompression compressedClause + /* * Non-keyword token types. These are hard-wired into the "flex" lexer. * They must be listed first so that their numeric codes do not depend on @@ -614,9 +617,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT - COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT - CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE - CROSS CSV CUBE CURRENT_P + COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT + CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY + COST CREATE CROSS CSV CUBE CURRENT_P CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE @@ -2168,6 +2171,15 @@ alter_table_cmd: n->missing_ok = true; $$ = (Node *)n; } + /* ALTER TABLE ALTER [COLUMN] SET (NOT COMPRESSED | COMPRESSED [WITH ()]) */ + | ALTER opt_column ColId SET columnCompression + { + AlterTableCmd *n = makeNode(AlterTableCmd); + n->subtype = AT_AlterColumnCompression; + n->name = $3; + n->def = $5; + $$ = (Node *)n; + } /* ALTER TABLE DROP [COLUMN] IF EXISTS [RESTRICT|CASCADE] */ | DROP opt_column IF_P EXISTS ColId opt_drop_behavior { @@ -3332,11 +3344,12 @@ TypedTableElement: | TableConstraint { $$ = $1; } ; -columnDef: ColId Typename create_generic_options ColQualList +columnDef: ColId Typename optColumnCompression create_generic_options ColQualList { ColumnDef *n = makeNode(ColumnDef); n->colname = $1; n->typeName = $2; + n->compression = (ColumnCompression *) $3; n->inhcount = 0; n->is_local = true; n->is_not_null = false; @@ -3346,8 +3359,8 @@ columnDef: ColId Typename create_generic_options ColQualList n->raw_default = NULL; n->cooked_default = NULL; n->collOid = InvalidOid; - n->fdwoptions = $3; - SplitColQualList($4, &n->constraints, &n->collClause, + n->fdwoptions = $4; + SplitColQualList($5, &n->constraints, &n->collClause, yyscanner); n->location = @1; $$ = (Node *)n; @@ -3394,6 +3407,37 @@ columnOptions: ColId ColQualList } ; +compressedClause: + COMPRESSED name optCompressionParameters + { + ColumnCompression *n = makeNode(ColumnCompression); + n->methodName = $2; + n->options = (List *) $3; + $$ = (Node *) n; + } + ; + +columnCompression: + compressedClause | + NOT COMPRESSED + { + ColumnCompression *n = makeNode(ColumnCompression); + n->methodName = NULL; + n->options = NIL; + $$ = (Node *) n; + } + ; + +optColumnCompression: + compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */ + | /*EMPTY*/ { $$ = NULL; } + ; + +optCompressionParameters: + WITH '(' generic_option_list ')' { $$ = $3; } + | /*EMPTY*/ { $$ = NIL; } + ; + ColQualList: ColQualList ColConstraint { $$ = lappend($1, $2); } | /*EMPTY*/ { $$ = NIL; } @@ -5754,6 +5798,15 @@ DefineStmt: n->if_not_exists = true; $$ = (Node *)n; } + | CREATE COMPRESSION METHOD any_name HANDLER handler_name + { + DefineStmt *n = makeNode(DefineStmt); + n->kind = OBJECT_COMPRESSION_METHOD; + n->args = NIL; + n->defnames = $4; + n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6)); + $$ = (Node *) n; + } ; definition: '(' def_list ')' { $$ = $2; } @@ -6262,6 +6315,7 @@ drop_type_any_name: /* object types taking name_list */ drop_type_name: ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; } + | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; } | EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; } | EXTENSION { $$ = OBJECT_EXTENSION; } | FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; } @@ -6325,7 +6379,7 @@ opt_restart_seqs: * The COMMENT ON statement can take different forms based upon the type of * the object associated with the comment. The form of the statement is: * - * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION | + * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION | * DATABASE | DOMAIN | * EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER | * FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE | @@ -6515,6 +6569,7 @@ comment_type_any_name: /* object types taking name */ comment_type_name: ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; } + | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; } | DATABASE { $$ = OBJECT_DATABASE; } | EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; } | EXTENSION { $$ = OBJECT_EXTENSION; } @@ -14704,6 +14759,8 @@ unreserved_keyword: | COMMENTS | COMMIT | COMMITTED + | COMPRESSED + | COMPRESSION | CONFIGURATION | CONFLICT | CONNECTION diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c index 8461da490a..39f21d4da7 100644 --- a/src/backend/parser/parse_utilcmd.c +++ b/src/backend/parser/parse_utilcmd.c @@ -27,6 +27,7 @@ #include "postgres.h" #include "access/amapi.h" +#include "access/compression.h" #include "access/htup_details.h" #include "access/reloptions.h" #include "catalog/dependency.h" @@ -494,6 +495,39 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column, *sname_p = sname; } +/* + * transformColumnCompression + * + * Build ALTER TABLE .. SET COMPRESSED command if a compression method was + * specified for the column + */ +void +transformColumnCompression(ColumnDef *column, RangeVar *relation, + AlterTableStmt **alterStmt) +{ + if (column->compression) + { + AlterTableCmd *cmd; + + cmd = makeNode(AlterTableCmd); + cmd->subtype = AT_AlterColumnCompression; + cmd->name = column->colname; + cmd->def = (Node *) column->compression; + cmd->behavior = DROP_RESTRICT; + cmd->missing_ok = false; + + if (!*alterStmt) + { + *alterStmt = makeNode(AlterTableStmt); + (*alterStmt)->relation = relation; + (*alterStmt)->relkind = OBJECT_TABLE; + (*alterStmt)->cmds = NIL; + } + + (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd); + } +} + /* * transformColumnDefinition - * transform a single ColumnDef within CREATE TABLE @@ -794,6 +828,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column) cxt->alist = lappend(cxt->alist, stmt); } + + if (cxt->isalter) + { + AlterTableStmt *stmt = NULL; + + transformColumnCompression(column, cxt->relation, &stmt); + + if (stmt) + cxt->alist = lappend(cxt->alist, stmt); + } } /* @@ -1003,6 +1047,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla def->collOid = attribute->attcollation; def->constraints = NIL; def->location = -1; + if (attribute->attcompression) + def->compression = GetColumnCompressionForAttribute(attribute); + else + def->compression = NULL; /* * Add to column list diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c index 0f607bab70..0ff169e200 100644 --- a/src/backend/replication/logical/reorderbuffer.c +++ b/src/backend/replication/logical/reorderbuffer.c @@ -2878,7 +2878,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn, VARSIZE(chunk) - VARHDRSZ); data_done += VARSIZE(chunk) - VARHDRSZ; } - Assert(data_done == toast_pointer.va_extsize); + Assert(data_done == VARATT_EXTERNAL_GET_SIZE(toast_pointer)); /* make sure its marked as compressed or not */ if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)) diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c index 82a707af7b..331d133660 100644 --- a/src/backend/tcop/utility.c +++ b/src/backend/tcop/utility.c @@ -998,6 +998,7 @@ ProcessUtilitySlow(ParseState *pstate, foreach(l, stmts) { Node *stmt = (Node *) lfirst(l); + Node *alterStmt = NULL; if (IsA(stmt, CreateStmt)) { @@ -1008,7 +1009,9 @@ ProcessUtilitySlow(ParseState *pstate, address = DefineRelation((CreateStmt *) stmt, RELKIND_RELATION, InvalidOid, NULL, - queryString); + queryString, + &alterStmt); + EventTriggerCollectSimpleCommand(address, secondaryObject, stmt); @@ -1042,7 +1045,8 @@ ProcessUtilitySlow(ParseState *pstate, address = DefineRelation((CreateStmt *) stmt, RELKIND_FOREIGN_TABLE, InvalidOid, NULL, - queryString); + queryString, + &alterStmt); CreateForeignTable((CreateForeignTableStmt *) stmt, address.objectId); EventTriggerCollectSimpleCommand(address, @@ -1074,6 +1078,9 @@ ProcessUtilitySlow(ParseState *pstate, NULL); } + if (alterStmt) + lappend(stmts, alterStmt); + /* Need CCI between commands */ if (lnext(l) != NULL) CommandCounterIncrement(); @@ -1283,6 +1290,11 @@ ProcessUtilitySlow(ParseState *pstate, stmt->definition, stmt->if_not_exists); break; + case OBJECT_COMPRESSION_METHOD: + Assert(stmt->args == NIL); + address = DefineCompressionMethod(stmt->defnames, + stmt->definition); + break; default: elog(ERROR, "unrecognized define stmt type: %d", (int) stmt->kind); @@ -1696,6 +1708,11 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel) case OBJECT_FOREIGN_TABLE: RemoveRelations(stmt); break; + case OBJECT_COMPRESSION_METHOD: + if (stmt->behavior == DROP_CASCADE) + { + /* TODO decompress columns instead of their deletion */ + } default: RemoveObjects(stmt); break; @@ -2309,6 +2326,9 @@ CreateCommandTag(Node *parsetree) case OBJECT_STATISTIC_EXT: tag = "DROP STATISTICS"; break; + case OBJECT_COMPRESSION_METHOD: + tag = "DROP COMPRESSION METHOD"; + break; default: tag = "???"; } @@ -2412,6 +2432,9 @@ CreateCommandTag(Node *parsetree) case OBJECT_ACCESS_METHOD: tag = "CREATE ACCESS METHOD"; break; + case OBJECT_COMPRESSION_METHOD: + tag = "CREATE COMPRESSION METHOD"; + break; default: tag = "???"; } diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c index be793539a3..a5cfbe3d4c 100644 --- a/src/backend/utils/adt/pseudotypes.c +++ b/src/backend/utils/adt/pseudotypes.c @@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal); PSEUDOTYPE_DUMMY_IO_FUNCS(opaque); PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement); PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray); +PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler); diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c index b0a9217d1e..e1314e1f2e 100644 --- a/src/backend/utils/adt/tsvector.c +++ b/src/backend/utils/adt/tsvector.c @@ -14,11 +14,14 @@ #include "postgres.h" +#include "access/compression.h" +#include "catalog/pg_type.h" #include "libpq/pqformat.h" #include "tsearch/ts_locale.h" #include "tsearch/ts_utils.h" #include "utils/builtins.h" #include "utils/memutils.h" +#include "common/pg_lzcompress.h" typedef struct { @@ -548,3 +551,85 @@ tsvectorrecv(PG_FUNCTION_ARGS) PG_RETURN_TSVECTOR(vec); } + +/* + * Compress tsvector using LZ compression. + * Instead of trying to compress whole tsvector we compress only text part + * here. This approach gives more compressibility for tsvectors. + */ +static struct varlena * +tsvector_compress(CompressionOptions * cmoptions, const struct varlena *data) +{ + char *tmp; + int32 valsize = VARSIZE_ANY_EXHDR(data); + int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED, + lenc; + + char *arr = VARDATA(data), + *str = STRPTR((TSVector) data); + int32 arrsize = str - arr; + + Assert(!VARATT_IS_COMPRESSED(data)); + tmp = palloc0(len); + + /* we try to compress string part of tsvector first */ + lenc = pglz_compress(str, + valsize - arrsize, + tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize, + PGLZ_strategy_default); + + if (lenc >= 0) + { + /* tsvector is compressible, copy size and entries to its beginning */ + memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize); + SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED); + return (struct varlena *) tmp; + } + + pfree(tmp); + return NULL; +} + +static struct varlena * +tsvector_decompress(CompressionOptions * cmoptions, const struct varlena *data) +{ + char *tmp, + *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED; + int32 count, + arrsize, + len = VARRAWSIZE_4B_C(data) + VARHDRSZ; + + Assert(VARATT_IS_CUSTOM_COMPRESSED(data)); + tmp = palloc0(len); + SET_VARSIZE(tmp, len); + count = *((uint32 *) raw_data); + arrsize = sizeof(uint32) + count * sizeof(WordEntry); + memcpy(VARDATA(tmp), raw_data, arrsize); + + if (pglz_decompress(raw_data + arrsize, + VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize, + VARDATA(tmp) + arrsize, + VARRAWSIZE_4B_C(data) - arrsize) < 0) + elog(ERROR, "compressed tsvector is corrupted"); + + return (struct varlena *) tmp; +} + +Datum +tsvector_compression_handler(PG_FUNCTION_ARGS) +{ + CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *) + PG_GETARG_POINTER(0); + CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine); + Oid typeid = opargs->typeid; + + if (OidIsValid(typeid) && typeid != TSVECTOROID) + elog(ERROR, "unexpected type %d for tsvector compression handler", typeid); + + cmr->configure = NULL; + cmr->drop = NULL; + cmr->compress = tsvector_compress; + cmr->decompress = tsvector_decompress; + + PG_RETURN_POINTER(cmr); +} diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 1908420d82..de834c5593 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -30,6 +30,7 @@ #include #include +#include "access/compression.h" #include "access/hash.h" #include "access/htup_details.h" #include "access/multixact.h" @@ -77,6 +78,7 @@ #include "storage/smgr.h" #include "utils/array.h" #include "utils/builtins.h" +#include "utils/datum.h" #include "utils/fmgroids.h" #include "utils/inval.h" #include "utils/lsyscache.h" diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index 888edbb325..c689f60e47 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -31,6 +31,8 @@ #include "catalog/pg_authid.h" #include "catalog/pg_cast.h" #include "catalog/pg_collation.h" +#include "catalog/pg_compression.h" +#include "catalog/pg_compression_opt.h" #include "catalog/pg_constraint.h" #include "catalog/pg_conversion.h" #include "catalog/pg_database.h" @@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = { }, 8 }, + {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */ + CompressionMethodOidIndexId, + 1, + { + ObjectIdAttributeNumber, + 0, + 0, + 0 + }, + 8 + }, + {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */ + CompressionMethodNameIndexId, + 1, + { + Anum_pg_compression_cmname, + 0, + 0, + 0 + }, + 8 + }, + {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */ + CompressionOptionsOidIndexId, + 1, + { + Anum_pg_compression_opt_cmoptoid, + 0, + 0, + 0 + }, + 8, + }, {ConversionRelationId, /* CONDEFAULT */ ConversionDefaultIndexId, 4, diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c index 4b47951de1..7ba6d31251 100644 --- a/src/bin/pg_dump/common.c +++ b/src/bin/pg_dump/common.c @@ -54,6 +54,7 @@ static DumpableObject **oprinfoindex; static DumpableObject **collinfoindex; static DumpableObject **nspinfoindex; static DumpableObject **extinfoindex; +static DumpableObject **cminfoindex; static int numTables; static int numTypes; static int numFuncs; @@ -61,6 +62,7 @@ static int numOperators; static int numCollations; static int numNamespaces; static int numExtensions; +static int numCompressionMethods; /* This is an array of object identities, not actual DumpableObjects */ static ExtensionMemberId *extmembers; @@ -93,6 +95,8 @@ getSchemaData(Archive *fout, int *numTablesPtr) NamespaceInfo *nspinfo; ExtensionInfo *extinfo; InhInfo *inhinfo; + CompressionMethodInfo *cminfo; + int numAggregates; int numInherits; int numRules; @@ -289,6 +293,11 @@ getSchemaData(Archive *fout, int *numTablesPtr) write_msg(NULL, "reading subscriptions\n"); getSubscriptions(fout); + if (g_verbose) + write_msg(NULL, "reading compression methods\n"); + cminfo = getCompressionMethods(fout, &numCompressionMethods); + cminfoindex = buildIndexArray(cminfo, numCompressionMethods, sizeof(CompressionMethodInfo)); + *numTablesPtr = numTables; return tblinfo; } @@ -827,6 +836,17 @@ findExtensionByOid(Oid oid) return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions); } +/* + * findCompressionMethodByOid + * finds the entry (in cminfo) of the compression method with the given oid + * returns NULL if not found + */ +CompressionMethodInfo * +findCompressionMethodByOid(Oid oid) +{ + return (CompressionMethodInfo *) findObjectByOid(oid, cminfoindex, + numCompressionMethods); +} /* * setExtensionMembership diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h index ce3100f09d..5c1aaad48e 100644 --- a/src/bin/pg_dump/pg_backup.h +++ b/src/bin/pg_dump/pg_backup.h @@ -77,6 +77,7 @@ typedef struct _restoreOptions int no_publications; /* Skip publication entries */ int no_security_labels; /* Skip security label entries */ int no_subscriptions; /* Skip subscription entries */ + int no_compression_methods; /* Skip compression methods */ int strict_names; const char *filename; @@ -121,6 +122,8 @@ typedef struct _restoreOptions bool *idWanted; /* array showing which dump IDs to emit */ int enable_row_security; int sequence_data; /* dump sequence data even in schema-only mode */ + int compression_options_data; /* dump compression options even + * in schema-only mode */ int binary_upgrade; } RestoreOptions; @@ -149,6 +152,7 @@ typedef struct _dumpOptions int no_security_labels; int no_publications; int no_subscriptions; + int no_compression_methods; int no_synchronized_snapshots; int no_unlogged_table_data; int serializable_deferrable; @@ -170,6 +174,8 @@ typedef struct _dumpOptions char *outputSuperuser; int sequence_data; /* dump sequence data even in schema-only mode */ + int compression_options_data; /* dump compression options even + * in schema-only mode */ } DumpOptions; /* diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c index ec2fa8b9b9..77eb55eb69 100644 --- a/src/bin/pg_dump/pg_backup_archiver.c +++ b/src/bin/pg_dump/pg_backup_archiver.c @@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt) dopt->include_everything = ropt->include_everything; dopt->enable_row_security = ropt->enable_row_security; dopt->sequence_data = ropt->sequence_data; + dopt->compression_options_data = ropt->compression_options_data; return dopt; } diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index d8fb356130..6df688adb8 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -361,6 +361,7 @@ main(int argc, char **argv) {"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1}, {"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1}, {"no-subscriptions", no_argument, &dopt.no_subscriptions, 1}, + {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1}, {"no-sync", no_argument, NULL, 7}, {NULL, 0, NULL, 0} @@ -581,6 +582,9 @@ main(int argc, char **argv) if (dopt.binary_upgrade) dopt.sequence_data = 1; + if (dopt.binary_upgrade) + dopt.compression_options_data = 1; + if (dopt.dataOnly && dopt.schemaOnly) { write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n"); @@ -785,6 +789,9 @@ main(int argc, char **argv) if (dopt.schemaOnly && dopt.sequence_data) getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE); + if (dopt.schemaOnly && dopt.compression_options_data) + getCompressionOptions(fout); + /* * In binary-upgrade mode, we do not have to worry about the actual blob * data or the associated metadata that resides in the pg_largeobject and @@ -3957,6 +3964,205 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo) destroyPQExpBuffer(query); } +/* + * getCompressionMethods + * get information about compression methods + */ +CompressionMethodInfo * +getCompressionMethods(Archive *fout, int *numMethods) +{ + DumpOptions *dopt = fout->dopt; + PQExpBuffer query; + PGresult *res; + CompressionMethodInfo *cminfo; + int i_tableoid; + int i_oid; + int i_handler; + int i_name; + int i, + ntups; + + if (dopt->no_compression_methods || fout->remoteVersion < 110000) + return NULL; + + query = createPQExpBuffer(); + resetPQExpBuffer(query); + + /* Get the compression methods in current database. */ + appendPQExpBuffer(query, "SELECT c.tableoid, c.oid, c.cmname, " + "c.cmhandler::pg_catalog.regproc as cmhandler " + "FROM pg_catalog.pg_compression c"); + res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); + + ntups = PQntuples(res); + + i_tableoid = PQfnumber(res, "tableoid"); + i_oid = PQfnumber(res, "oid"); + i_name = PQfnumber(res, "cmname"); + i_handler = PQfnumber(res, "cmhandler"); + + cminfo = pg_malloc(ntups * sizeof(CompressionMethodInfo)); + + for (i = 0; i < ntups; i++) + { + cminfo[i].dobj.objType = DO_COMPRESSION_METHOD; + cminfo[i].dobj.catId.tableoid = + atooid(PQgetvalue(res, i, i_tableoid)); + cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid)); + AssignDumpId(&cminfo[i].dobj); + cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler)); + cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name)); + + /* Decide whether we want to dump it */ + selectDumpableObject(&(cminfo[i].dobj), fout); + } + if (numMethods) + *numMethods = ntups; + + PQclear(res); + destroyPQExpBuffer(query); + + return cminfo; +} + +/* + * dumpCompressionMethod + * dump the definition of the given compression method + */ +static void +dumpCompressionMethod(Archive *fout, CompressionMethodInfo * cminfo) +{ + PQExpBuffer query; + + if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION)) + return; + + /* Make sure we are in proper schema */ + selectSourceSchema(fout, "pg_catalog"); + + query = createPQExpBuffer(); + appendPQExpBuffer(query, "CREATE COMPRESSION METHOD %s HANDLER", + fmtId(cminfo->dobj.name)); + appendPQExpBuffer(query, " %s;\n", cminfo->cmhandler); + + ArchiveEntry(fout, + cminfo->dobj.catId, + cminfo->dobj.dumpId, + cminfo->dobj.name, + NULL, + NULL, + "", false, + "COMPRESSION METHOD", SECTION_PRE_DATA, + query->data, "", NULL, + NULL, 0, + NULL, NULL); + + destroyPQExpBuffer(query); +} + +/* + * getCompressionOptions + * get information about compression options. + * this information should be migrated at binary upgrade + */ +void +getCompressionOptions(Archive *fout) +{ + PQExpBuffer query; + PGresult *res; + CompressionOptionsInfo *cminfo; + int i_tableoid; + int i_oid; + int i_handler; + int i_name; + int i_options; + int i, + ntups; + + if (fout->remoteVersion < 110000) + return; + + query = createPQExpBuffer(); + resetPQExpBuffer(query); + + appendPQExpBuffer(query, + "SELECT c.tableoid, c.cmoptoid, c.cmname," + " c.cmhandler::pg_catalog.regproc as cmhandler, c.cmoptions " + "FROM pg_catalog.pg_compression_opt c"); + res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); + + ntups = PQntuples(res); + + i_tableoid = PQfnumber(res, "tableoid"); + i_oid = PQfnumber(res, "cmoptoid"); + i_name = PQfnumber(res, "cmname"); + i_handler = PQfnumber(res, "cmhandler"); + i_options = PQfnumber(res, "cmoptions"); + + cminfo = pg_malloc(ntups * sizeof(CompressionOptionsInfo)); + + for (i = 0; i < ntups; i++) + { + cminfo[i].dobj.objType = DO_COMPRESSION_OPTIONS; + cminfo[i].dobj.catId.tableoid = + atooid(PQgetvalue(res, i, i_tableoid)); + cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid)); + AssignDumpId(&cminfo[i].dobj); + cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler)); + cminfo[i].cmoptions = pg_strdup(PQgetvalue(res, i, i_options)); + cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name)); + cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION; + } + + PQclear(res); + destroyPQExpBuffer(query); +} + +/* + * dumpCompressionOptions + * dump the given compression options + */ +static void +dumpCompressionOptions(Archive *fout, CompressionOptionsInfo * cminfo) +{ + PQExpBuffer query; + + if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION)) + return; + + /* Make sure we are in proper schema */ + selectSourceSchema(fout, "pg_catalog"); + + query = createPQExpBuffer(); + if (strlen(cminfo->cmoptions) > 0) + appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler," + " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), '%s');\n", + cminfo->dobj.catId.oid, + cminfo->dobj.name, + cminfo->cmhandler, + cminfo->cmoptions); + else + appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler," + " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), NULL);\n", + cminfo->dobj.catId.oid, + cminfo->dobj.name, + cminfo->cmhandler); + + ArchiveEntry(fout, + cminfo->dobj.catId, + cminfo->dobj.dumpId, + cminfo->dobj.name, + NULL, + NULL, + "", false, + "COMPRESSION OPTIONS", SECTION_PRE_DATA, + query->data, "", NULL, + NULL, 0, + NULL, NULL); + + destroyPQExpBuffer(query); +} + static void binary_upgrade_set_type_oids_by_type_oid(Archive *fout, PQExpBuffer upgrade_buffer, @@ -4484,7 +4690,47 @@ getTypes(Archive *fout, int *numTypes) /* Make sure we are in proper schema */ selectSourceSchema(fout, "pg_catalog"); - if (fout->remoteVersion >= 90600) + if (fout->remoteVersion >= 110000) + { + PQExpBuffer acl_subquery = createPQExpBuffer(); + PQExpBuffer racl_subquery = createPQExpBuffer(); + PQExpBuffer initacl_subquery = createPQExpBuffer(); + PQExpBuffer initracl_subquery = createPQExpBuffer(); + + buildACLQueries(acl_subquery, racl_subquery, initacl_subquery, + initracl_subquery, "t.typacl", "t.typowner", "'T'", + dopt->binary_upgrade); + + appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, " + "t.typnamespace, " + "%s AS typacl, " + "%s AS rtypacl, " + "%s AS inittypacl, " + "%s AS initrtypacl, " + "(%s t.typowner) AS rolname, " + "t.typelem, t.typrelid, " + "CASE WHEN t.typrelid = 0 THEN ' '::\"char\" " + "ELSE (SELECT relkind FROM pg_class WHERE oid = t.typrelid) END AS typrelkind, " + "t.typtype, t.typisdefined, " + "t.typname[0] = '_' AND t.typelem != 0 AND " + "(SELECT typarray FROM pg_type te WHERE oid = t.typelem) = t.oid AS isarray " + "FROM pg_type t " + "LEFT JOIN pg_init_privs pip ON " + "(t.oid = pip.objoid " + "AND pip.classoid = 'pg_type'::regclass " + "AND pip.objsubid = 0) ", + acl_subquery->data, + racl_subquery->data, + initacl_subquery->data, + initracl_subquery->data, + username_subquery); + + destroyPQExpBuffer(acl_subquery); + destroyPQExpBuffer(racl_subquery); + destroyPQExpBuffer(initacl_subquery); + destroyPQExpBuffer(initracl_subquery); + } + else if (fout->remoteVersion >= 90600) { PQExpBuffer acl_subquery = createPQExpBuffer(); PQExpBuffer racl_subquery = createPQExpBuffer(); @@ -7895,6 +8141,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) int i_attoptions; int i_attcollation; int i_attfdwoptions; + int i_attcmoptions; + int i_attcmname; PGresult *res; int ntups; bool hasdefaults; @@ -7930,7 +8178,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) resetPQExpBuffer(q); - if (fout->remoteVersion >= 100000) + if (fout->remoteVersion >= 110000) + { + /* + * attcompression is new in version 11 + */ + appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, " + "a.attstattarget, a.attstorage, t.typstorage, " + "a.attnotnull, a.atthasdef, a.attisdropped, " + "a.attlen, a.attalign, a.attislocal, " + "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, " + "array_to_string(a.attoptions, ', ') AS attoptions, " + "CASE WHEN a.attcollation <> t.typcollation " + "THEN a.attcollation ELSE 0 END AS attcollation, " + "a.attidentity, " + /* fdw options */ + "pg_catalog.array_to_string(ARRAY(" + "SELECT pg_catalog.quote_ident(option_name) || " + "' ' || pg_catalog.quote_literal(option_value) " + "FROM pg_catalog.pg_options_to_table(attfdwoptions) " + "ORDER BY option_name" + "), E',\n ') AS attfdwoptions, " + /* compression options */ + "pg_catalog.array_to_string(ARRAY(" + "SELECT pg_catalog.quote_ident(option_name) || " + "' ' || pg_catalog.quote_literal(option_value) " + "FROM pg_catalog.pg_options_to_table(c.cmoptions) " + "ORDER BY option_name" + "), E',\n ') AS attcmoptions, " + "c.cmname AS attcmname " + /* FROM */ + "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t " + "ON a.atttypid = t.oid " + "LEFT JOIN pg_catalog.pg_compression_opt c " + "ON a.attcompression = c.cmoptoid " + "WHERE a.attrelid = '%u'::pg_catalog.oid " + "AND a.attnum > 0::pg_catalog.int2 " + "ORDER BY a.attnum", + tbinfo->dobj.catId.oid); + } + else if (fout->remoteVersion >= 100000) { /* * attidentity is new in version 10. @@ -7949,9 +8236,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) "' ' || pg_catalog.quote_literal(option_value) " "FROM pg_catalog.pg_options_to_table(attfdwoptions) " "ORDER BY option_name" - "), E',\n ') AS attfdwoptions " + "), E',\n ') AS attfdwoptions, " + "NULL AS attcmoptions, " + "NULL AS attcmname " "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t " "ON a.atttypid = t.oid " + "LEFT JOIN pg_catalog.pg_compression_opt c " + "ON a.attcompression = c.oid " "WHERE a.attrelid = '%u'::pg_catalog.oid " "AND a.attnum > 0::pg_catalog.int2 " "ORDER BY a.attnum", @@ -7975,7 +8266,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) "' ' || pg_catalog.quote_literal(option_value) " "FROM pg_catalog.pg_options_to_table(attfdwoptions) " "ORDER BY option_name" - "), E',\n ') AS attfdwoptions " + "), E',\n ') AS attfdwoptions, " + "NULL AS attcmoptions, " + "NULL AS attcmname " "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t " "ON a.atttypid = t.oid " "WHERE a.attrelid = '%u'::pg_catalog.oid " @@ -7999,7 +8292,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) "array_to_string(a.attoptions, ', ') AS attoptions, " "CASE WHEN a.attcollation <> t.typcollation " "THEN a.attcollation ELSE 0 END AS attcollation, " - "NULL AS attfdwoptions " + "NULL AS attfdwoptions, " + "NULL AS attcmoptions, " + "NULL AS attcmname " "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t " "ON a.atttypid = t.oid " "WHERE a.attrelid = '%u'::pg_catalog.oid " @@ -8017,7 +8312,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, " "array_to_string(a.attoptions, ', ') AS attoptions, " "0 AS attcollation, " - "NULL AS attfdwoptions " + "NULL AS attfdwoptions, " + "NULL AS attcmoptions, " + "NULL AS attcmname " "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t " "ON a.atttypid = t.oid " "WHERE a.attrelid = '%u'::pg_catalog.oid " @@ -8034,7 +8331,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) "a.attlen, a.attalign, a.attislocal, " "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, " "'' AS attoptions, 0 AS attcollation, " - "NULL AS attfdwoptions " + "NULL AS attfdwoptions, " + "NULL AS attcmoptions, " + "NULL AS attcmname " "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t " "ON a.atttypid = t.oid " "WHERE a.attrelid = '%u'::pg_catalog.oid " @@ -8064,6 +8363,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) i_attoptions = PQfnumber(res, "attoptions"); i_attcollation = PQfnumber(res, "attcollation"); i_attfdwoptions = PQfnumber(res, "attfdwoptions"); + i_attcmname = PQfnumber(res, "attcmname"); + i_attcmoptions = PQfnumber(res, "attcmoptions"); tbinfo->numatts = ntups; tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *)); @@ -8080,6 +8381,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *)); tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid)); tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *)); + tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *)); + tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *)); tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool)); tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool)); tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *)); @@ -8107,6 +8410,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions)); tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation)); tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions)); + tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions)); + tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname)); tbinfo->attrdefs[j] = NULL; /* fix below */ if (PQgetvalue(res, j, i_atthasdef)[0] == 't') hasdefaults = true; @@ -9596,6 +9901,12 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj) case DO_SUBSCRIPTION: dumpSubscription(fout, (SubscriptionInfo *) dobj); break; + case DO_COMPRESSION_METHOD: + dumpCompressionMethod(fout, (CompressionMethodInfo *) dobj); + break; + case DO_COMPRESSION_OPTIONS: + dumpCompressionOptions(fout, (CompressionOptionsInfo *) dobj); + break; case DO_PRE_DATA_BOUNDARY: case DO_POST_DATA_BOUNDARY: /* never dumped, nothing to do */ @@ -15513,6 +15824,15 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo) } } + if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])) + { + appendPQExpBuffer(q, " COMPRESSED %s", + tbinfo->attcmnames[j]); + if (nonemptyReloptions(tbinfo->attcmoptions[j])) + appendPQExpBuffer(q, " WITH (%s)", + tbinfo->attcmoptions[j]); + } + if (has_default) appendPQExpBuffer(q, " DEFAULT %s", tbinfo->attrdefs[j]->adef_expr); @@ -17778,6 +18098,8 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs, case DO_FOREIGN_SERVER: case DO_TRANSFORM: case DO_BLOB: + case DO_COMPRESSION_METHOD: + case DO_COMPRESSION_OPTIONS: /* Pre-data objects: must come before the pre-data boundary */ addObjectDependency(preDataBound, dobj->dumpId); break; diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h index da884ffd09..88b18ba15c 100644 --- a/src/bin/pg_dump/pg_dump.h +++ b/src/bin/pg_dump/pg_dump.h @@ -83,7 +83,9 @@ typedef enum DO_POLICY, DO_PUBLICATION, DO_PUBLICATION_REL, - DO_SUBSCRIPTION + DO_SUBSCRIPTION, + DO_COMPRESSION_METHOD, + DO_COMPRESSION_OPTIONS } DumpableObjectType; /* component types of an object which can be selected for dumping */ @@ -315,6 +317,8 @@ typedef struct _tableInfo char **attoptions; /* per-attribute options */ Oid *attcollation; /* per-attribute collation selection */ char **attfdwoptions; /* per-attribute fdw options */ + char **attcmoptions; /* per-attribute compression options */ + char **attcmnames; /* per-attribute compression method names */ bool *notnull; /* NOT NULL constraints on attributes */ bool *inhNotNull; /* true if NOT NULL is inherited */ struct _attrDefInfo **attrdefs; /* DEFAULT expressions */ @@ -611,6 +615,21 @@ typedef struct _SubscriptionInfo char *subpublications; } SubscriptionInfo; +/* The CompressionMethodInfo struct is used to represent compression method */ +typedef struct _CompressionMethodInfo +{ + DumpableObject dobj; + char *cmhandler; +} CompressionMethodInfo; + +/* The CompressionOptionsInfo struct is used to represent compression options */ +typedef struct _CompressionOptionsInfo +{ + DumpableObject dobj; + char *cmhandler; + char *cmoptions; +} CompressionOptionsInfo; + /* * We build an array of these with an entry for each object that is an * extension member according to pg_depend. @@ -654,6 +673,7 @@ extern OprInfo *findOprByOid(Oid oid); extern CollInfo *findCollationByOid(Oid oid); extern NamespaceInfo *findNamespaceByOid(Oid oid); extern ExtensionInfo *findExtensionByOid(Oid oid); +extern CompressionMethodInfo * findCompressionMethodByOid(Oid oid); extern void setExtensionMembership(ExtensionMemberId *extmems, int nextmems); extern ExtensionInfo *findOwningExtension(CatalogId catalogId); @@ -711,5 +731,8 @@ extern void getPublications(Archive *fout); extern void getPublicationTables(Archive *fout, TableInfo tblinfo[], int numTables); extern void getSubscriptions(Archive *fout); +extern CompressionMethodInfo * getCompressionMethods(Archive *fout, + int *numMethods); +void getCompressionOptions(Archive *fout); #endif /* PG_DUMP_H */ diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c index 48b6dd594c..f5db0280e2 100644 --- a/src/bin/pg_dump/pg_dump_sort.c +++ b/src/bin/pg_dump/pg_dump_sort.c @@ -80,7 +80,9 @@ static const int dbObjectTypePriority[] = 34, /* DO_POLICY */ 35, /* DO_PUBLICATION */ 36, /* DO_PUBLICATION_REL */ - 37 /* DO_SUBSCRIPTION */ + 37, /* DO_SUBSCRIPTION */ + 17, /* DO_COMPRESSION_METHOD */ + 17 /* DO_COMPRESSION_OPTIONS */ }; static DumpId preDataBoundId; @@ -1436,6 +1438,16 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize) "POST-DATA BOUNDARY (ID %d)", obj->dumpId); return; + case DO_COMPRESSION_METHOD: + snprintf(buf, bufsize, + "COMPRESSION METHOD %s (ID %d OID %u)", + obj->name, obj->dumpId, obj->catId.oid); + return; + case DO_COMPRESSION_OPTIONS: + snprintf(buf, bufsize, + "COMPRESSION OPTIONS %s (ID %d OID %u)", + obj->name, obj->dumpId, obj->catId.oid); + return; } /* shouldn't get here */ snprintf(buf, bufsize, diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c index 41c5ff89b7..6b72ccf9ea 100644 --- a/src/bin/pg_dump/pg_dumpall.c +++ b/src/bin/pg_dump/pg_dumpall.c @@ -77,6 +77,7 @@ static int use_setsessauth = 0; static int no_publications = 0; static int no_security_labels = 0; static int no_subscriptions = 0; +static int no_compression_methods = 0; static int no_unlogged_table_data = 0; static int no_role_passwords = 0; static int server_version; @@ -137,6 +138,7 @@ main(int argc, char *argv[]) {"no-role-passwords", no_argument, &no_role_passwords, 1}, {"no-security-labels", no_argument, &no_security_labels, 1}, {"no-subscriptions", no_argument, &no_subscriptions, 1}, + {"no-compression-methods", no_argument, &no_compression_methods, 1}, {"no-sync", no_argument, NULL, 4}, {"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1}, @@ -405,6 +407,8 @@ main(int argc, char *argv[]) appendPQExpBufferStr(pgdumpopts, " --no-security-labels"); if (no_subscriptions) appendPQExpBufferStr(pgdumpopts, " --no-subscriptions"); + if (no_compression_methods) + appendPQExpBufferStr(pgdumpopts, " --no-compression-methods"); if (no_unlogged_table_data) appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data"); @@ -628,6 +632,7 @@ help(void) printf(_(" --no-role-passwords do not dump passwords for roles\n")); printf(_(" --no-security-labels do not dump security label assignments\n")); printf(_(" --no-subscriptions do not dump subscriptions\n")); + printf(_(" --no-compression-methods do not dump compression methods\n")); printf(_(" --no-sync do not wait for changes to be written safely to disk\n")); printf(_(" --no-tablespaces do not dump tablespace assignments\n")); printf(_(" --no-unlogged-table-data do not dump unlogged table data\n")); diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c index 860a211a3c..810403ec9c 100644 --- a/src/bin/pg_dump/pg_restore.c +++ b/src/bin/pg_dump/pg_restore.c @@ -74,6 +74,7 @@ main(int argc, char **argv) static int no_publications = 0; static int no_security_labels = 0; static int no_subscriptions = 0; + static int no_compression_methods = 0; static int strict_names = 0; struct option cmdopts[] = { @@ -122,6 +123,7 @@ main(int argc, char **argv) {"no-publications", no_argument, &no_publications, 1}, {"no-security-labels", no_argument, &no_security_labels, 1}, {"no-subscriptions", no_argument, &no_subscriptions, 1}, + {"no-compression-methods", no_argument, &no_compression_methods, 1}, {NULL, 0, NULL, 0} }; @@ -361,6 +363,7 @@ main(int argc, char **argv) opts->no_publications = no_publications; opts->no_security_labels = no_security_labels; opts->no_subscriptions = no_subscriptions; + opts->no_compression_methods = no_compression_methods; if (if_exists && !opts->dropSchema) { diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c index 8cc4de3878..8b0dbfa45c 100644 --- a/src/bin/psql/command.c +++ b/src/bin/psql/command.c @@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd) success = listConversions(pattern, show_verbose, show_system); break; case 'C': - success = listCasts(pattern, show_verbose); + if (cmd[2] == 'M') + success = describeCompressionMethods(pattern, show_verbose); + else + success = listCasts(pattern, show_verbose); break; case 'd': if (strncmp(cmd, "ddp", 3) == 0) diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c index b7b978a361..49160ded0c 100644 --- a/src/bin/psql/describe.c +++ b/src/bin/psql/describe.c @@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose) return true; } +/* + * \dCM + * Takes an optional regexp to select particular compression methods + */ +bool +describeCompressionMethods(const char *pattern, bool verbose) +{ + PQExpBufferData buf; + PGresult *res; + printQueryOpt myopt = pset.popt; + static const bool translate_columns[] = {false, false, false}; + + if (pset.sversion < 100000) + { + char sverbuf[32]; + + psql_error("The server (version %s) does not support compression methods.\n", + formatPGVersionNumber(pset.sversion, false, + sverbuf, sizeof(sverbuf))); + return true; + } + + initPQExpBuffer(&buf); + + printfPQExpBuffer(&buf, + "SELECT cmname AS \"%s\"", + gettext_noop("Name")); + + if (verbose) + { + appendPQExpBuffer(&buf, + ",\n cmhandler AS \"%s\",\n" + " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"", + gettext_noop("Handler"), + gettext_noop("Description")); + } + + appendPQExpBufferStr(&buf, + "\nFROM pg_catalog.pg_compression\n"); + + processSQLNamePattern(pset.db, &buf, pattern, false, false, + NULL, "cmname", NULL, + NULL); + + appendPQExpBufferStr(&buf, "ORDER BY 1;"); + + res = PSQLexec(buf.data); + termPQExpBuffer(&buf); + if (!res) + return false; + + myopt.nullPrint = NULL; + myopt.title = _("List of compression methods"); + myopt.translate_header = true; + myopt.translate_columns = translate_columns; + myopt.n_translate_columns = lengthof(translate_columns); + + printQuery(res, &myopt, pset.queryFout, false, pset.logfile); + + PQclear(res); + return true; +} + /* * \db * Takes an optional regexp to select particular tablespaces @@ -1716,6 +1779,22 @@ describeOneTableDetails(const char *schemaname, if (verbose) { appendPQExpBufferStr(&buf, ",\n a.attstorage"); + + /* compresssion info */ + if (pset.sversion >= 110000) + appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE " + " (SELECT c.cmname || " + " (CASE WHEN cmoptions IS NULL " + " THEN '' " + " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)" + " FROM pg_options_to_table(cmoptions)), ', ') || ')'" + " END) " + " FROM pg_catalog.pg_compression_opt c " + " WHERE c.cmoptoid = a.attcompression) " + " END AS attcmname"); + else + appendPQExpBufferStr(&buf, "\n NULL AS attcmname"); + appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget"); /* @@ -1830,6 +1909,10 @@ describeOneTableDetails(const char *schemaname, if (verbose) { headers[cols++] = gettext_noop("Storage"); + + if (tableinfo.relkind == RELKIND_RELATION) + headers[cols++] = gettext_noop("Compression"); + if (tableinfo.relkind == RELKIND_RELATION || tableinfo.relkind == RELKIND_INDEX || tableinfo.relkind == RELKIND_MATVIEW || @@ -1925,6 +2008,11 @@ describeOneTableDetails(const char *schemaname, "???")))), false, false); + /* Column compression. */ + if (tableinfo.relkind == RELKIND_RELATION) + printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1), + false, false); + /* Statistics target, if the relkind supports this feature */ if (tableinfo.relkind == RELKIND_RELATION || tableinfo.relkind == RELKIND_INDEX || @@ -1932,7 +2020,7 @@ describeOneTableDetails(const char *schemaname, tableinfo.relkind == RELKIND_FOREIGN_TABLE || tableinfo.relkind == RELKIND_PARTITIONED_TABLE) { - printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1), + printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2), false, false); } @@ -1943,7 +2031,7 @@ describeOneTableDetails(const char *schemaname, tableinfo.relkind == RELKIND_COMPOSITE_TYPE || tableinfo.relkind == RELKIND_FOREIGN_TABLE || tableinfo.relkind == RELKIND_PARTITIONED_TABLE) - printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2), + printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3), false, false); } } diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h index 14a5667f3e..0ab8518a36 100644 --- a/src/bin/psql/describe.h +++ b/src/bin/psql/describe.h @@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose); /* \db */ extern bool describeTablespaces(const char *pattern, bool verbose); +/* \dCM */ +extern bool describeCompressionMethods(const char *pattern, bool verbose); + /* \df, \dfa, \dfn, \dft, \dfw, etc. */ extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem); diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c index a926c40b9b..25d2fbc125 100644 --- a/src/bin/psql/help.c +++ b/src/bin/psql/help.c @@ -227,6 +227,7 @@ slashUsage(unsigned short int pager) fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n")); fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n")); fprintf(output, _(" \\dC[+] [PATTERN] list casts\n")); + fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n")); fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n")); fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n")); fprintf(output, _(" \\ddp [PATTERN] list default privileges\n")); diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index b3e3799c13..a323ca80c6 100644 --- a/src/bin/psql/tab-complete.c +++ b/src/bin/psql/tab-complete.c @@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = { " AND d.datname = pg_catalog.current_database() "\ " AND s.subdbid = d.oid" +#define Query_for_list_of_compression_methods \ +" SELECT pg_catalog.quote_ident(cmname) "\ +" FROM pg_catalog.pg_compression "\ +" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'" + /* the silly-looking length condition is just to eat up the current word */ #define Query_for_list_of_arguments \ "SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\ @@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = { * CREATE CONSTRAINT TRIGGER is not supported here because it is designed * to be used only by pg_dump. */ + {"COMPRESSION METHOD", NULL, NULL}, {"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW}, {"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"}, {"DATABASE", Query_for_list_of_databases}, @@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end) "\\a", "\\connect", "\\conninfo", "\\C", "\\cd", "\\copy", "\\copyright", "\\crosstabview", - "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD", - "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df", + "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp", + "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df", "\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL", "\\dm", "\\dn", "\\do", "\\dO", "\\dp", "\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS", @@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end) /* ALTER TABLE ALTER [COLUMN] SET */ else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) - COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE"); + COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE"); /* ALTER TABLE ALTER [COLUMN] SET ( */ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") || Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "(")) COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited"); + else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") || + Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED")) + COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods); + else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) || + Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny)) + COMPLETE_WITH_CONST("WITH ("); /* ALTER TABLE ALTER [COLUMN] SET STORAGE */ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") || Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE")) @@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end) "SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION", "TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION", "OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT", - "TABLESPACE", "TEXT SEARCH", "ROLE", NULL}; + "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL}; COMPLETE_WITH_LIST(list_COMMENT); } else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD")) COMPLETE_WITH_QUERY(Query_for_list_of_access_methods); + else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD")) + COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods); else if (Matches3("COMMENT", "ON", "FOREIGN")) COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE"); else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH")) @@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end) else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny)) COMPLETE_WITH_CONST("HANDLER"); + /* CREATE COMPRESSION METHOD */ + /* Complete "CREATE COMPRESSION METHOD " */ + else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny)) + COMPLETE_WITH_CONST("HANDLER"); + /* Complete "CREATE COMPRESSION METHOD HANDLER" */ + else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER")) + COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL); + /* CREATE DATABASE */ else if (Matches3("CREATE", "DATABASE", MatchAny)) COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE", @@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end) Matches4("DROP", "ACCESS", "METHOD", MatchAny) || (Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) && ends_with(prev_wd, ')')) || + Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) || Matches4("DROP", "EVENT", "TRIGGER", MatchAny) || Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) || Matches4("DROP", "FOREIGN", "TABLE", MatchAny) || @@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end) else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny)) COMPLETE_WITH_LIST2("CASCADE", "RESTRICT"); + /* DROP COMPRESSION METHOD */ + else if (Matches2("DROP", "COMPRESSION")) + COMPLETE_WITH_CONST("METHOD"); + else if (Matches3("DROP", "COMPRESSION", "METHOD")) + COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods); + /* EXECUTE */ else if (Matches1("EXECUTE")) COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements); @@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end) COMPLETE_WITH_QUERY(Query_for_list_of_access_methods); else if (TailMatchesCS1("\\db*")) COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces); + else if (TailMatchesCS1("\\dCM*")) + COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods); else if (TailMatchesCS1("\\dD*")) COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL); else if (TailMatchesCS1("\\des*")) diff --git a/src/include/access/compression.h b/src/include/access/compression.h new file mode 100644 index 0000000000..6c0e5fda5f --- /dev/null +++ b/src/include/access/compression.h @@ -0,0 +1,66 @@ +/*------------------------------------------------------------------------- + * + * compression.h + * API for Postgres compression methods. + * + * Copyright (c) 2015-2016, PostgreSQL Global Development Group + * + * src/include/access/compression.h + * + *------------------------------------------------------------------------- + */ + +#ifndef COMPRESSION_H +#define COMPRESSION_H + +#include "postgres.h" +#include "catalog/pg_attribute.h" +#include "nodes/nodes.h" +#include "nodes/pg_list.h" + +/* parsenodes.h */ +typedef struct ColumnCompression ColumnCompression; +typedef struct CompressionMethodRoutine CompressionMethodRoutine; + +typedef struct +{ + Oid cmoptoid; + CompressionMethodRoutine *routine; + List *options; +} CompressionOptions; + +typedef void (*CompressionConfigureRoutine) + (Form_pg_attribute attr, List *options); +typedef struct varlena *(*CompressionRoutine) + (CompressionOptions * cmoptions, const struct varlena *data); + +/* + * API struct for a compression method. + * Note this must be stored in a single palloc'd chunk of memory. + */ +typedef struct CompressionMethodRoutine +{ + NodeTag type; + + CompressionConfigureRoutine configure; + CompressionConfigureRoutine drop; + CompressionRoutine compress; + CompressionRoutine decompress; +} CompressionMethodRoutine; + +/* Compression method handler parameters */ +typedef struct CompressionMethodOpArgs +{ + Oid cmhanderid; + Oid typeid; +} CompressionMethodOpArgs; + +extern CompressionMethodRoutine * GetCompressionRoutine(Oid cmoptoid); +extern List *GetCompressionOptionsList(Oid cmoptoid); +extern Oid CreateCompressionOptions(Form_pg_attribute attr, Oid cmid, + List *options); +extern ColumnCompression * GetColumnCompressionForAttribute(Form_pg_attribute att); +extern void CheckCompressionMismatch(ColumnCompression * c1, + ColumnCompression * c2, const char *attributeName); + +#endif /* COMPRESSION_H */ diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h index cd43e3a52e..2e1c92123e 100644 --- a/src/include/access/reloptions.h +++ b/src/include/access/reloptions.h @@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des extern Datum transformRelOptions(Datum oldOptions, List *defList, const char *namspace, char *validnsps[], bool ignoreOids, bool isReset); -extern List *untransformRelOptions(Datum options); extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc, amoptions_function amoptions); extern relopt_value *parseRelOptions(Datum options, bool validate, @@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize, relopt_value *options, int numoptions, bool validate, const relopt_parse_elt *elems, int nelems); +extern char *formatRelOptions(List *options); +extern List *untransformRelOptions(Datum options); extern bytea *default_reloptions(Datum reloptions, bool validate, relopt_kind kind); diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h index 2be5af1d3e..867adc1bd8 100644 --- a/src/include/access/tupdesc.h +++ b/src/include/access/tupdesc.h @@ -14,7 +14,9 @@ #ifndef TUPDESC_H #define TUPDESC_H +#include "postgres.h" #include "access/attnum.h" +#include "access/compression.h" #include "catalog/pg_attribute.h" #include "nodes/pg_list.h" @@ -88,6 +90,9 @@ typedef struct tupleDesc /* Accessor for the i'th attribute of tupdesc. */ #define TupleDescAttr(tupdesc, i) (&(tupdesc)->attrs[(i)]) +#define TupleDescAttrCompression(tupdesc, i) \ + ((tupdesc)->tdcompression? &((tupdesc)->tdcompression[i]) : NULL) + extern TupleDesc CreateTemplateTupleDesc(int natts, bool hasoid); diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h index fd9f83ac44..0632588e5d 100644 --- a/src/include/access/tuptoaster.h +++ b/src/include/access/tuptoaster.h @@ -101,6 +101,15 @@ /* Size of an EXTERNAL datum that contains an indirection pointer */ #define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect)) +/* + * va_extinfo in varatt_external contains actual length of the external data + * and optional flags + */ +#define VARATT_EXTERNAL_GET_SIZE(toast_pointer) \ + ((toast_pointer).va_extinfo & 0x3FFFFFFF) +#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \ + (((toast_pointer).va_extinfo >> 30) == 0x02) + /* * Testing whether an externally-stored value is compressed now requires * comparing extsize (the actual length of the external data) to rawsize @@ -109,7 +118,7 @@ * saves space, so we expect either equality or less-than. */ #define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \ - ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ) + (VARATT_EXTERNAL_GET_SIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ) /* * Macro to fetch the possibly-unaligned contents of an EXTERNAL datum @@ -210,7 +219,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc, * Create a compressed version of a varlena datum, if possible * ---------- */ -extern Datum toast_compress_datum(Datum value); +extern Datum toast_compress_datum(Datum value, Oid cmoptoid); /* ---------- * toast_raw_datum_size - diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h index b9f98423cc..36cadd409e 100644 --- a/src/include/catalog/dependency.h +++ b/src/include/catalog/dependency.h @@ -165,10 +165,12 @@ typedef enum ObjectClass OCLASS_PUBLICATION, /* pg_publication */ OCLASS_PUBLICATION_REL, /* pg_publication_rel */ OCLASS_SUBSCRIPTION, /* pg_subscription */ - OCLASS_TRANSFORM /* pg_transform */ + OCLASS_TRANSFORM, /* pg_transform */ + OCLASS_COMPRESSION_METHOD, /* pg_compression */ + OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */ } ObjectClass; -#define LAST_OCLASS OCLASS_TRANSFORM +#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS /* flag bits for performDeletion/performMultipleDeletions: */ #define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */ diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h index ef8493674c..7fe9f1f059 100644 --- a/src/include/catalog/indexing.h +++ b/src/include/catalog/indexing.h @@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops)); #define CollationOidIndexId 3085 +DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops)); +#define CompressionMethodOidIndexId 3422 +DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops)); +#define CompressionMethodNameIndexId 3423 + +DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(cmoptoid oid_ops)); +#define CompressionOptionsOidIndexId 3424 + DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops)); #define ConstraintNameNspIndexId 2664 DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops)); diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h index bcf28e8f04..caadd61031 100644 --- a/src/include/catalog/pg_attribute.h +++ b/src/include/catalog/pg_attribute.h @@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK /* attribute's collation */ Oid attcollation; + /* attribute's compression options or InvalidOid */ + Oid attcompression; + #ifdef CATALOG_VARLEN /* variable-length fields start here */ /* NOTE: The following fields are not present in tuple descriptors. */ @@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK * ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout, * guaranteed-not-null part of a pg_attribute row. This is in fact as much * of the row as gets copied into tuple descriptors, so don't expect you - * can access fields beyond attcollation except in a real tuple! + * can access fields beyond attcompression except in a real tuple! */ #define ATTRIBUTE_FIXED_PART_SIZE \ - (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid)) + (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid)) /* ---------------- * Form_pg_attribute corresponds to a pointer to a tuple with @@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute; * ---------------- */ -#define Natts_pg_attribute 22 -#define Anum_pg_attribute_attrelid 1 -#define Anum_pg_attribute_attname 2 -#define Anum_pg_attribute_atttypid 3 -#define Anum_pg_attribute_attstattarget 4 -#define Anum_pg_attribute_attlen 5 -#define Anum_pg_attribute_attnum 6 -#define Anum_pg_attribute_attndims 7 -#define Anum_pg_attribute_attcacheoff 8 -#define Anum_pg_attribute_atttypmod 9 -#define Anum_pg_attribute_attbyval 10 -#define Anum_pg_attribute_attstorage 11 -#define Anum_pg_attribute_attalign 12 -#define Anum_pg_attribute_attnotnull 13 -#define Anum_pg_attribute_atthasdef 14 -#define Anum_pg_attribute_attidentity 15 -#define Anum_pg_attribute_attisdropped 16 -#define Anum_pg_attribute_attislocal 17 -#define Anum_pg_attribute_attinhcount 18 -#define Anum_pg_attribute_attcollation 19 -#define Anum_pg_attribute_attacl 20 -#define Anum_pg_attribute_attoptions 21 -#define Anum_pg_attribute_attfdwoptions 22 +#define Natts_pg_attribute 23 +#define Anum_pg_attribute_attrelid 1 +#define Anum_pg_attribute_attname 2 +#define Anum_pg_attribute_atttypid 3 +#define Anum_pg_attribute_attstattarget 4 +#define Anum_pg_attribute_attlen 5 +#define Anum_pg_attribute_attnum 6 +#define Anum_pg_attribute_attndims 7 +#define Anum_pg_attribute_attcacheoff 8 +#define Anum_pg_attribute_atttypmod 9 +#define Anum_pg_attribute_attbyval 10 +#define Anum_pg_attribute_attstorage 11 +#define Anum_pg_attribute_attalign 12 +#define Anum_pg_attribute_attnotnull 13 +#define Anum_pg_attribute_atthasdef 14 +#define Anum_pg_attribute_attidentity 15 +#define Anum_pg_attribute_attisdropped 16 +#define Anum_pg_attribute_attislocal 17 +#define Anum_pg_attribute_attinhcount 18 +#define Anum_pg_attribute_attcollation 19 +#define Anum_pg_attribute_attcompression 20 +#define Anum_pg_attribute_attacl 21 +#define Anum_pg_attribute_attoptions 22 +#define Anum_pg_attribute_attfdwoptions 23 /* ---------------- diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h index b256657bda..40f5cc4f18 100644 --- a/src/include/catalog/pg_class.h +++ b/src/include/catalog/pg_class.h @@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class; */ DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_)); DESCR(""); -DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_)); +DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_)); DESCR(""); DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_)); DESCR(""); diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h new file mode 100644 index 0000000000..1d5f9ac479 --- /dev/null +++ b/src/include/catalog/pg_compression.h @@ -0,0 +1,55 @@ +/*------------------------------------------------------------------------- + * + * pg_compression.h + * definition of the system "compression method" relation (pg_compression) + * along with the relation's initial contents. + * + * + * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/catalog/pg_compression.h + * + * NOTES + * the genbki.pl script reads this file and generates .bki + * information from the DATA() statements. + * + * XXX do NOT break up DATA() statements into multiple lines! + * the scripts are not as smart as you might think... + * + *------------------------------------------------------------------------- + */ +#ifndef PG_COMPRESSION_H +#define PG_COMPRESSION_H + +#include "catalog/genbki.h" + +/* ---------------- + * pg_compression definition. cpp turns this into + * typedef struct FormData_pg_compression + * ---------------- + */ +#define CompressionMethodRelationId 3419 + +CATALOG(pg_compression,3419) +{ + NameData cmname; /* compression method name */ + regproc cmhandler; /* compression handler */ +} FormData_pg_compression; + +/* ---------------- + * Form_pg_compression corresponds to a pointer to a tuple with + * the format of pg_compression relation. + * ---------------- + */ +typedef FormData_pg_compression * Form_pg_compression; + +/* ---------------- + * compiler constants for pg_compression + * ---------------- + */ +#define Natts_pg_compression 2 +#define Anum_pg_compression_cmname 1 +#define Anum_pg_compression_cmhandler 2 + +#endif /* PG_COMPRESSION_H */ diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h new file mode 100644 index 0000000000..ddcef814a7 --- /dev/null +++ b/src/include/catalog/pg_compression_opt.h @@ -0,0 +1,56 @@ +/*------------------------------------------------------------------------- + * + * pg_compression_opt.h + * + * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/catalog/pg_compression_opt.h + * + * NOTES + * the genbki.pl script reads this file and generates .bki + * information from the DATA() statements. + * + * XXX do NOT break up DATA() statements into multiple lines! + * the scripts are not as smart as you might think... + * + *------------------------------------------------------------------------- + */ +#ifndef PG_COMPRESSION_OPT_H +#define PG_COMPRESSION_OPT_H + +#include "catalog/genbki.h" + +/* ---------------- + * pg_compression_opt definition. cpp turns this into + * typedef struct FormData_pg_compression_opt + * ---------------- + */ +#define CompressionOptRelationId 3420 + +CATALOG(pg_compression_opt,3420) BKI_WITHOUT_OIDS +{ + Oid cmoptoid; /* compression options oid */ + NameData cmname; /* name of compression method */ + regproc cmhandler; /* compression handler */ + text cmoptions[1]; /* specific options from WITH */ +} FormData_pg_compression_opt; + +/* ---------------- + * Form_pg_compression_opt corresponds to a pointer to a tuple with + * the format of pg_compression_opt relation. + * ---------------- + */ +typedef FormData_pg_compression_opt * Form_pg_compression_opt; + +/* ---------------- + * compiler constants for pg_compression_opt + * ---------------- + */ +#define Natts_pg_compression_opt 4 +#define Anum_pg_compression_opt_cmoptoid 1 +#define Anum_pg_compression_opt_cmname 2 +#define Anum_pg_compression_opt_cmhandler 3 +#define Anum_pg_compression_opt_cmoptions 4 + +#endif /* PG_COMPRESSION_OPT_H */ diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h index c969375981..dc3fa66f2e 100644 --- a/src/include/catalog/pg_proc.h +++ b/src/include/catalog/pg_proc.h @@ -3881,6 +3881,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s DESCR("I/O"); DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ )); DESCR("I/O"); +DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ )); +DESCR("I/O"); +DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ )); +DESCR("I/O"); /* tablesample method handlers */ DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ )); @@ -4677,6 +4681,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s DESCR("I/O"); DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ )); DESCR("I/O"); +DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ )); +DESCR("tsvector compression handler"); DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ )); DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ )); diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h index e3551440a0..ec8c3df953 100644 --- a/src/include/catalog/pg_type.h +++ b/src/include/catalog/pg_type.h @@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han #define TSM_HANDLEROID 3310 DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ )); #define ANYRANGEOID 3831 +DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ )); +#define COMPRESSION_HANDLEROID 3421 /* diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h index bfead9af3d..a98ecc12ce 100644 --- a/src/include/commands/defrem.h +++ b/src/include/commands/defrem.h @@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt); extern void RemoveUserMappingById(Oid umId); extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid); extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt); +extern Datum optionListToArray(List *options); extern Datum transformGenericOptions(Oid catalogId, Datum oldOptions, List *options, @@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok); extern Oid get_am_oid(const char *amname, bool missing_ok); extern char *get_am_name(Oid amOid); +/* commands/compressioncmds.c */ +extern ObjectAddress DefineCompressionMethod(List *names, List *parameters); +extern void RemoveCompressionMethodById(Oid cmOid); +extern void RemoveCompressionOptionsById(Oid cmoptoid); +extern Oid get_compression_method_oid(const char *cmname, bool missing_ok); +extern char *get_compression_method_name(Oid cmOid); +extern char *get_compression_method_name_for_opt(Oid cmoptoid); + /* support routines in commands/define.c */ extern char *defGetString(DefElem *def); diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index da3ff5dbee..c50d9525d0 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -24,7 +24,8 @@ extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, - ObjectAddress *typaddress, const char *queryString); + ObjectAddress *typaddress, const char *queryString, + Node **pAlterStmt); extern void RemoveRelations(DropStmt *drop); diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h index ffeeb4919b..6dc49a73b6 100644 --- a/src/include/nodes/nodes.h +++ b/src/include/nodes/nodes.h @@ -468,6 +468,7 @@ typedef enum NodeTag T_PartitionBoundSpec, T_PartitionRangeDatum, T_PartitionCmd, + T_ColumnCompression, T_VacuumRelation, /* @@ -498,7 +499,8 @@ typedef enum NodeTag T_FdwRoutine, /* in foreign/fdwapi.h */ T_IndexAmRoutine, /* in access/amapi.h */ T_TsmRoutine, /* in access/tsmapi.h */ - T_ForeignKeyCacheInfo /* in utils/rel.h */ + T_ForeignKeyCacheInfo, /* in utils/rel.h */ + T_CompressionMethodRoutine, /* in access/compression.h */ } NodeTag; /* diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 80c19b2a55..9fa905ae6e 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -615,6 +615,19 @@ typedef struct RangeTableSample int location; /* method name location, or -1 if unknown */ } RangeTableSample; +/* + * ColumnCompression - compression parameters for some attribute + * + * This represents compression information defined using clause: + * .. COMPRESSED WITH () + */ +typedef struct ColumnCompression +{ + NodeTag type; + char *methodName; + List *options; +} ColumnCompression; + /* * ColumnDef - column definition (used in various creates) * @@ -638,6 +651,7 @@ typedef struct ColumnDef NodeTag type; char *colname; /* name of column */ TypeName *typeName; /* type of column */ + ColumnCompression *compression; int inhcount; /* number of times column is inherited */ bool is_local; /* column has local (non-inherited) def'n */ bool is_not_null; /* NOT NULL constraint specified? */ @@ -1622,6 +1636,7 @@ typedef enum ObjectType OBJECT_CAST, OBJECT_COLUMN, OBJECT_COLLATION, + OBJECT_COMPRESSION_METHOD, OBJECT_CONVERSION, OBJECT_DATABASE, OBJECT_DEFAULT, @@ -1769,7 +1784,8 @@ typedef enum AlterTableType AT_DetachPartition, /* DETACH PARTITION */ AT_AddIdentity, /* ADD IDENTITY */ AT_SetIdentity, /* SET identity column options */ - AT_DropIdentity /* DROP IDENTITY */ + AT_DropIdentity, /* DROP IDENTITY */ + AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */ } AlterTableType; typedef struct ReplicaIdentityStmt diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h index f50e45e886..7bfc6e6be4 100644 --- a/src/include/parser/kwlist.h +++ b/src/include/parser/kwlist.h @@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD) PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD) PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD) PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD) +PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD) +PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD) PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD) PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD) PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD) diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h index e749432ef0..5cab77457a 100644 --- a/src/include/parser/parse_utilcmd.h +++ b/src/include/parser/parse_utilcmd.h @@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt, const char *queryString); extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt, const char *queryString); -extern void transformRuleStmt(RuleStmt *stmt, const char *queryString, +void transformRuleStmt(RuleStmt *stmt, const char *queryString, List **actions, Node **whereClause); extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt); extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent, PartitionBoundSpec *spec); +extern void transformColumnCompression(ColumnDef *column, RangeVar *relation, + AlterTableStmt **alterStmt); #endif /* PARSE_UTILCMD_H */ diff --git a/src/include/postgres.h b/src/include/postgres.h index 1ca9b60ea1..d21974696f 100644 --- a/src/include/postgres.h +++ b/src/include/postgres.h @@ -56,7 +56,7 @@ /* * struct varatt_external is a traditional "TOAST pointer", that is, the * information needed to fetch a Datum stored out-of-line in a TOAST table. - * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ. + * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ. * This struct must not contain any padding, because we sometimes compare * these pointers using memcmp. * @@ -68,7 +68,8 @@ typedef struct varatt_external { int32 va_rawsize; /* Original data size (includes header) */ - int32 va_extsize; /* External saved size (doesn't) */ + uint32 va_extinfo; /* External saved size (without header) and + * flags */ Oid va_valueid; /* Unique ID of value within TOAST table */ Oid va_toastrelid; /* RelID of TOAST table containing it */ } varatt_external; @@ -146,9 +147,18 @@ typedef union struct /* Compressed-in-line format */ { uint32 va_header; - uint32 va_rawsize; /* Original data size (excludes header) */ + uint32 va_info; /* Original data size (excludes header) and + * flags */ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */ } va_compressed; + struct /* Compressed-in-line format */ + { + uint32 va_header; + uint32 va_info; /* Original data size (excludes header) and + * flags */ + Oid va_cmoptoid; /* Oid of compression options */ + char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */ + } va_custom_compressed; } varattrib_4b; typedef struct @@ -281,8 +291,14 @@ typedef struct #define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data) #define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data) +/* va_info in va_compress contains raw size of datum and optional flags */ #define VARRAWSIZE_4B_C(PTR) \ - (((varattrib_4b *) (PTR))->va_compressed.va_rawsize) + (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF) +#define VARFLAGS_4B_C(PTR) \ + (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30) + +#define VARHDRSZ_CUSTOM_COMPRESSED \ + (offsetof(varattrib_4b, va_custom_compressed.va_data)) /* Externally visible macros */ @@ -311,6 +327,8 @@ typedef struct #define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR) #define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR) +#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \ + (VARFLAGS_4B_C(PTR) == 0x02)) #define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR) #define VARATT_IS_EXTERNAL_ONDISK(PTR) \ (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK) diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h index 254a811aff..6ad889af7a 100644 --- a/src/include/utils/acl.h +++ b/src/include/utils/acl.h @@ -210,6 +210,7 @@ typedef enum AclObjectKind ACL_KIND_EXTENSION, /* pg_extension */ ACL_KIND_PUBLICATION, /* pg_publication */ ACL_KIND_SUBSCRIPTION, /* pg_subscription */ + ACL_KIND_COMPRESSION_METHOD, /* pg_compression */ MAX_ACL_KIND /* MUST BE LAST */ } AclObjectKind; diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h index 8a0be41929..ff7cb530fd 100644 --- a/src/include/utils/syscache.h +++ b/src/include/utils/syscache.h @@ -48,6 +48,9 @@ enum SysCacheIdentifier CLAOID, COLLNAMEENCNSP, COLLOID, + COMPRESSIONMETHODOID, + COMPRESSIONMETHODNAME, + COMPRESSIONOPTIONSOID, CONDEFAULT, CONNAMENSP, CONSTROID, diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out index 65e9c626b3..112f0eda47 100644 --- a/src/test/regress/expected/copy2.out +++ b/src/test/regress/expected/copy2.out @@ -438,10 +438,10 @@ begin end $$ language plpgsql immutable; alter table check_con_tbl add check (check_con_function(check_con_tbl.*)); \d+ check_con_tbl - Table "public.check_con_tbl" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - f1 | integer | | | | plain | | + Table "public.check_con_tbl" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + f1 | integer | | | | plain | | | Check constraints: "check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*)) diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out new file mode 100644 index 0000000000..16d66a0263 --- /dev/null +++ b/src/test/regress/expected/create_cm.out @@ -0,0 +1,145 @@ +-- test drop +CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler; +CREATE TABLE droptest(fts tsvector COMPRESSED ts1); +DROP COMPRESSION METHOD ts1; +ERROR: cannot drop compression method ts1 because other objects depend on it +DETAIL: compression options for ts1 depends on compression method ts1 +table droptest column fts depends on compression options for ts1 +HINT: Use DROP ... CASCADE to drop the dependent objects too. +DROP COMPRESSION METHOD ts1 CASCADE; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to compression options for ts1 +drop cascades to table droptest column fts +\d+ droptest + Table "public.droptest" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------+-----------+----------+---------+---------+-------------+--------------+------------- + +DROP TABLE droptest; +CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler; +CREATE TABLE cmtest(fts tsvector COMPRESSED ts1); +SELECT * FROM pg_compression; + cmname | cmhandler +--------+------------------------------ + ts1 | tsvector_compression_handler +(1 row) + +SELECT cmhandler, cmoptions FROM pg_compression_opt; + cmhandler | cmoptions +------------------------------+----------- + tsvector_compression_handler | +(1 row) + +\dCM +List of compression methods + Name +------ + ts1 +(1 row) + +\d+ cmtest + Table "public.cmtest" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+----------+-----------+----------+---------+----------+-------------+--------------+------------- + fts | tsvector | | | | extended | ts1 | | + +INSERT INTO cmtest + SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' ')) + FROM generate_series(1,200) i; +SELECT length(fts) FROM cmtest; + length +-------- + 200 +(1 row) + +SELECT length(fts) FROM cmtest; + length +-------- + 200 +(1 row) + +-- check ALTER commands +ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED; +\d+ cmtest + Table "public.cmtest" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+----------+-----------+----------+---------+----------+-------------+--------------+------------- + fts | tsvector | | | | extended | | | + +ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz'); +ERROR: the compression method "ts1" does not take any options +ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1; +\d+ cmtest + Table "public.cmtest" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+----------+-----------+----------+---------+----------+-------------+--------------+------------- + fts | tsvector | | | | extended | ts1 | | + +-- create different types of tables +SELECT * INTO cmtest2 FROM cmtest; +CREATE TABLE cmtest3 (LIKE cmtest); +CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest); +NOTICE: merging column "fts" with inherited definition +CREATE TABLE cmtest5(fts tsvector); +-- we update usual datum with compressed datum +INSERT INTO cmtest5 + SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' ')) + FROM generate_series(1,200) i; +UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest; +\d+ cmtest3 + Table "public.cmtest3" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+----------+-----------+----------+---------+----------+-------------+--------------+------------- + fts | tsvector | | | | extended | ts1 | | + +\d+ cmtest4 + Table "public.cmtest4" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+----------+-----------+----------+---------+----------+-------------+--------------+------------- + fts | tsvector | | | | extended | ts1 | | + a | integer | | | | plain | | | +Inherits: cmtest + +DROP TABLE cmtest CASCADE; +NOTICE: drop cascades to table cmtest4 +DROP TABLE cmtest3; +SELECT cmhandler, cmoptions FROM pg_compression_opt; + cmhandler | cmoptions +------------------------------+----------- + tsvector_compression_handler | + tsvector_compression_handler | + tsvector_compression_handler | + tsvector_compression_handler | +(4 rows) + +DROP COMPRESSION METHOD ts1 CASCADE; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to compression options for ts1 +drop cascades to compression options for ts1 +drop cascades to compression options for ts1 +drop cascades to compression options for ts1 +SELECT * FROM pg_compression; + cmname | cmhandler +--------+----------- +(0 rows) + +SELECT * FROM pg_compression_opt; + cmoptoid | cmname | cmhandler | cmoptions +----------+--------+-----------+----------- +(0 rows) + +-- check that moved tuples still can be decompressed +SELECT length(fts) FROM cmtest2; + length +-------- + 200 +(1 row) + +SELECT length(fts) FROM cmtest5; + length +-------- + 200 +(1 row) + +DROP TABLE cmtest2; +DROP TABLE cmtest5; diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out index 335cd37e18..b511daf9fa 100644 --- a/src/test/regress/expected/create_table.out +++ b/src/test/regress/expected/create_table.out @@ -590,10 +590,10 @@ CREATE TABLE oids_parted ( ) PARTITION BY RANGE (a) WITH OIDS; CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS; \d+ part_forced_oids - Table "public.part_forced_oids" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - a | integer | | | | plain | | + Table "public.part_forced_oids" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + a | integer | | | | plain | | | Partition of: oids_parted FOR VALUES FROM (1) TO (10) Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10)) Has OIDs: yes @@ -729,11 +729,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10); -- Partition bound in describe output \d+ part_b - Table "public.part_b" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | not null | 1 | plain | | + Table "public.part_b" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | not null | 1 | plain | | | Partition of: parted FOR VALUES IN ('b') Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text]))) Check constraints: @@ -756,11 +756,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10) -- a level-2 partition's constraint will include the parent's expressions \d+ part_c_1_10 - Table "public.part_c_1_10" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | not null | 0 | plain | | + Table "public.part_c_1_10" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | not null | 0 | plain | | | Partition of: part_c FOR VALUES FROM (1) TO (10) Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10)) Check constraints: @@ -793,46 +793,46 @@ Number of partitions: 3 (Use \d+ to list them.) CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c); CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE); \d+ unbounded_range_part - Table "public.unbounded_range_part" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - a | integer | | | | plain | | - b | integer | | | | plain | | - c | integer | | | | plain | | + Table "public.unbounded_range_part" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + a | integer | | | | plain | | | + b | integer | | | | plain | | | + c | integer | | | | plain | | | Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE) Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL)) DROP TABLE unbounded_range_part; CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE); \d+ range_parted4_1 - Table "public.range_parted4_1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - a | integer | | | | plain | | - b | integer | | | | plain | | - c | integer | | | | plain | | + Table "public.range_parted4_1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + a | integer | | | | plain | | | + b | integer | | | | plain | | | + c | integer | | | | plain | | | Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE) Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1)) CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE); \d+ range_parted4_2 - Table "public.range_parted4_2" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - a | integer | | | | plain | | - b | integer | | | | plain | | - c | integer | | | | plain | | + Table "public.range_parted4_2" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + a | integer | | | | plain | | | + b | integer | | | | plain | | | + c | integer | | | | plain | | | Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE) Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7)))) CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE); \d+ range_parted4_3 - Table "public.range_parted4_3" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - a | integer | | | | plain | | - b | integer | | | | plain | | - c | integer | | | | plain | | + Table "public.range_parted4_3" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + a | integer | | | | plain | | | + b | integer | | | | plain | | | + c | integer | | | | plain | | | Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE) Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9)) diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out index 3f405c94ce..b5ca8b820b 100644 --- a/src/test/regress/expected/create_table_like.out +++ b/src/test/regress/expected/create_table_like.out @@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text); ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL; CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE); \d+ ctlt12_storage - Table "public.ctlt12_storage" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+------+-----------+----------+---------+----------+--------------+------------- - a | text | | not null | | main | | - b | text | | | | extended | | - c | text | | | | external | | + Table "public.ctlt12_storage" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | not null | | main | | | + b | text | | | | extended | | | + c | text | | | | external | | | CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS); \d+ ctlt12_comments - Table "public.ctlt12_comments" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+------+-----------+----------+---------+----------+--------------+------------- - a | text | | not null | | extended | | A - b | text | | | | extended | | B - c | text | | | | extended | | C + Table "public.ctlt12_comments" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | not null | | extended | | | A + b | text | | | | extended | | | B + c | text | | | | extended | | | C CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1); NOTICE: merging column "a" with inherited definition NOTICE: merging column "b" with inherited definition NOTICE: merging constraint "ctlt1_a_check" with inherited definition \d+ ctlt1_inh - Table "public.ctlt1_inh" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+------+-----------+----------+---------+----------+--------------+------------- - a | text | | not null | | main | | A - b | text | | | | extended | | B + Table "public.ctlt1_inh" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | not null | | main | | | A + b | text | | | | extended | | | B Check constraints: "ctlt1_a_check" CHECK (length(a) > 2) Inherits: ctlt1 @@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3); NOTICE: merging multiple inherited definitions of column "a" \d+ ctlt13_inh - Table "public.ctlt13_inh" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+------+-----------+----------+---------+----------+--------------+------------- - a | text | | not null | | main | | - b | text | | | | extended | | - c | text | | | | external | | + Table "public.ctlt13_inh" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | not null | | main | | | + b | text | | | | extended | | | + c | text | | | | external | | | Check constraints: "ctlt1_a_check" CHECK (length(a) > 2) "ctlt3_a_check" CHECK (length(a) < 5) @@ -210,12 +210,12 @@ Inherits: ctlt1, CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1); NOTICE: merging column "a" with inherited definition \d+ ctlt13_like - Table "public.ctlt13_like" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+------+-----------+----------+---------+----------+--------------+------------- - a | text | | not null | | main | | A3 - b | text | | | | extended | | - c | text | | | | external | | C + Table "public.ctlt13_like" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | not null | | main | | | A3 + b | text | | | | extended | | | + c | text | | | | external | | | C Check constraints: "ctlt1_a_check" CHECK (length(a) > 2) "ctlt3_a_check" CHECK (length(a) < 5) @@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL); \d+ ctlt_all - Table "public.ctlt_all" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+------+-----------+----------+---------+----------+--------------+------------- - a | text | | not null | | main | | A - b | text | | | | extended | | B + Table "public.ctlt_all" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | not null | | main | | | A + b | text | | | | extended | | | B Indexes: "ctlt_all_pkey" PRIMARY KEY, btree (a) "ctlt_all_b_idx" btree (b) diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out index f4eebb75cf..2ac75c44b5 100644 --- a/src/test/regress/expected/domain.out +++ b/src/test/regress/expected/domain.out @@ -272,10 +272,10 @@ explain (verbose, costs off) create rule silly as on delete to dcomptable do instead update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0; \d+ dcomptable - Table "public.dcomptable" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+-----------+-----------+----------+---------+----------+--------------+------------- - d1 | dcomptype | | | | extended | | + Table "public.dcomptable" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+-----------+-----------+----------+---------+----------+-------------+--------------+------------- + d1 | dcomptype | | | | extended | | | Indexes: "dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1) Rules: @@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1 where d1[1].i > 0; \d+ dcomptable - Table "public.dcomptable" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+------------+-----------+----------+---------+----------+--------------+------------- - d1 | dcomptypea | | | | extended | | + Table "public.dcomptable" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+------------+-----------+----------+---------+----------+-------------+--------------+------------- + d1 | dcomptypea | | | | extended | | | Indexes: "dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1) Rules: diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out index 331f7a911f..530ce1b1d0 100644 --- a/src/test/regress/expected/foreign_data.out +++ b/src/test/regress/expected/foreign_data.out @@ -1330,12 +1330,12 @@ CREATE TABLE pt1 ( CREATE FOREIGN TABLE ft2 () INHERITS (pt1) SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value'); \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Child tables: ft2 \d+ ft2 @@ -1351,12 +1351,12 @@ Inherits: pt1 DROP FOREIGN TABLE ft2; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | CREATE FOREIGN TABLE ft2 ( c1 integer NOT NULL, @@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value') ALTER FOREIGN TABLE ft2 INHERIT pt1; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Child tables: ft2 \d+ ft2 @@ -1418,12 +1418,12 @@ Child tables: ct3, ft3 \d+ ct3 - Table "public.ct3" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.ct3" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Inherits: ft2 \d+ ft3 @@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer; ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL; ALTER TABLE pt1 ADD COLUMN c8 integer; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | | - c2 | text | | | | extended | | - c3 | date | | | | plain | | - c4 | integer | | | | plain | | - c5 | integer | | | 0 | plain | | - c6 | integer | | | | plain | | - c7 | integer | | not null | | plain | | - c8 | integer | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | + c4 | integer | | | | plain | | | + c5 | integer | | | 0 | plain | | | + c6 | integer | | | | plain | | | + c7 | integer | | not null | | plain | | | + c8 | integer | | | | plain | | | Child tables: ft2 \d+ ft2 @@ -1475,17 +1475,17 @@ Child tables: ct3, ft3 \d+ ct3 - Table "public.ct3" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | | - c2 | text | | | | extended | | - c3 | date | | | | plain | | - c4 | integer | | | | plain | | - c5 | integer | | | 0 | plain | | - c6 | integer | | | | plain | | - c7 | integer | | not null | | plain | | - c8 | integer | | | | plain | | + Table "public.ct3" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | + c4 | integer | | | | plain | | | + c5 | integer | | | 0 | plain | | | + c6 | integer | | | | plain | | | + c7 | integer | | not null | | plain | | | + c8 | integer | | | | plain | | | Inherits: ft2 \d+ ft3 @@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100); ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1; ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | - c4 | integer | | | 0 | plain | | - c5 | integer | | | | plain | | - c6 | integer | | not null | | plain | | - c7 | integer | | | | plain | | - c8 | text | | | | external | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | + c4 | integer | | | 0 | plain | | | + c5 | integer | | | | plain | | | + c6 | integer | | not null | | plain | | | + c7 | integer | | | | plain | | | + c8 | text | | | | external | | | Child tables: ft2 \d+ ft2 @@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6; ALTER TABLE pt1 DROP COLUMN c7; ALTER TABLE pt1 DROP COLUMN c8; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Child tables: ft2 \d+ ft2 @@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit -- child does not inherit NO INHERIT constraints \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Check constraints: "pt1chk1" CHECK (c1 > 0) NO INHERIT "pt1chk2" CHECK (c2 <> ''::text) @@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> ''); ALTER FOREIGN TABLE ft2 INHERIT pt1; -- child does not inherit NO INHERIT constraints \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Check constraints: "pt1chk1" CHECK (c1 > 0) NO INHERIT "pt1chk2" CHECK (c2 <> ''::text) @@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE; INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date); ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Check constraints: "pt1chk3" CHECK (c2 <> ''::text) NOT VALID Child tables: ft2 @@ -1694,12 +1694,12 @@ Inherits: pt1 -- VALIDATE CONSTRAINT need do nothing on foreign tables ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Check constraints: "pt1chk3" CHECK (c2 <> ''::text) Child tables: ft2 @@ -1721,12 +1721,12 @@ Inherits: pt1 -- OID system column ALTER TABLE pt1 SET WITH OIDS; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Check constraints: "pt1chk3" CHECK (c2 <> ''::text) Child tables: ft2 @@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR ERROR: cannot drop inherited column "oid" ALTER TABLE pt1 SET WITHOUT OIDS; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - c1 | integer | | not null | | plain | 10000 | - c2 | text | | | | extended | | - c3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + c1 | integer | | not null | | plain | | 10000 | + c2 | text | | | | extended | | | + c3 | date | | | | plain | | | Check constraints: "pt1chk3" CHECK (c2 <> ''::text) Child tables: ft2 @@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3; -- changes name of a constraint recursively ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check; \d+ pt1 - Table "public.pt1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - f1 | integer | | not null | | plain | 10000 | - f2 | text | | | | extended | | - f3 | date | | | | plain | | + Table "public.pt1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + f1 | integer | | not null | | plain | | 10000 | + f2 | text | | | | extended | | | + f3 | date | | | | plain | | | Check constraints: "f2_check" CHECK (f2 <> ''::text) Child tables: ft2 diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out index c698faff2f..a147b8217f 100644 --- a/src/test/regress/expected/inherit.out +++ b/src/test/regress/expected/inherit.out @@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed ERROR: cannot rename inherited column "aa" ALTER TABLE inhts RENAME d TO dd; \d+ inhts - Table "public.inhts" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - aa | integer | | | | plain | | - b | integer | | | | plain | | - c | integer | | | | plain | | - dd | integer | | | | plain | | + Table "public.inhts" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + aa | integer | | | | plain | | | + b | integer | | | | plain | | | + c | integer | | | | plain | | | + dd | integer | | | | plain | | | Inherits: inht1, inhs1 @@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa" NOTICE: merging multiple inherited definitions of column "b" ALTER TABLE inht1 RENAME aa TO aaa; \d+ inht4 - Table "public.inht4" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - aaa | integer | | | | plain | | - b | integer | | | | plain | | - x | integer | | | | plain | | - y | integer | | | | plain | | - z | integer | | | | plain | | + Table "public.inht4" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + aaa | integer | | | | plain | | | + b | integer | | | | plain | | | + x | integer | | | | plain | | | + y | integer | | | | plain | | | + z | integer | | | | plain | | | Inherits: inht2, inht3 @@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa; ALTER TABLE inht1 RENAME b TO bb; -- to be failed ERROR: cannot rename inherited column "b" \d+ inhts - Table "public.inhts" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - aaaa | integer | | | | plain | | - b | integer | | | | plain | | - x | integer | | | | plain | | - c | integer | | | | plain | | - d | integer | | | | plain | | + Table "public.inhts" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + aaaa | integer | | | | plain | | | + b | integer | | | | plain | | | + x | integer | | | | plain | | | + c | integer | | | | plain | | | + d | integer | | | | plain | | | Inherits: inht2, inhs1 @@ -1106,33 +1106,33 @@ drop cascades to table inht4 CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2)); CREATE TABLE test_constraints_inh () INHERITS (test_constraints); \d+ test_constraints - Table "public.test_constraints" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+-------------------+-----------+----------+---------+----------+--------------+------------- - id | integer | | | | plain | | - val1 | character varying | | | | extended | | - val2 | integer | | | | plain | | + Table "public.test_constraints" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+-------------------+-----------+----------+---------+----------+-------------+--------------+------------- + id | integer | | | | plain | | | + val1 | character varying | | | | extended | | | + val2 | integer | | | | plain | | | Indexes: "test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2) Child tables: test_constraints_inh ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key; \d+ test_constraints - Table "public.test_constraints" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+-------------------+-----------+----------+---------+----------+--------------+------------- - id | integer | | | | plain | | - val1 | character varying | | | | extended | | - val2 | integer | | | | plain | | + Table "public.test_constraints" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+-------------------+-----------+----------+---------+----------+-------------+--------------+------------- + id | integer | | | | plain | | | + val1 | character varying | | | | extended | | | + val2 | integer | | | | plain | | | Child tables: test_constraints_inh \d+ test_constraints_inh - Table "public.test_constraints_inh" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+-------------------+-----------+----------+---------+----------+--------------+------------- - id | integer | | | | plain | | - val1 | character varying | | | | extended | | - val2 | integer | | | | plain | | + Table "public.test_constraints_inh" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+-------------------+-----------+----------+---------+----------+-------------+--------------+------------- + id | integer | | | | plain | | | + val1 | character varying | | | | extended | | | + val2 | integer | | | | plain | | | Inherits: test_constraints DROP TABLE test_constraints_inh; @@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints ( ); CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints); \d+ test_ex_constraints - Table "public.test_ex_constraints" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+--------+-----------+----------+---------+---------+--------------+------------- - c | circle | | | | plain | | + Table "public.test_ex_constraints" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+--------+-----------+----------+---------+---------+-------------+--------------+------------- + c | circle | | | | plain | | | Indexes: "test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&) Child tables: test_ex_constraints_inh ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl; \d+ test_ex_constraints - Table "public.test_ex_constraints" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+--------+-----------+----------+---------+---------+--------------+------------- - c | circle | | | | plain | | + Table "public.test_ex_constraints" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+--------+-----------+----------+---------+---------+-------------+--------------+------------- + c | circle | | | | plain | | | Child tables: test_ex_constraints_inh \d+ test_ex_constraints_inh - Table "public.test_ex_constraints_inh" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+--------+-----------+----------+---------+---------+--------------+------------- - c | circle | | | | plain | | + Table "public.test_ex_constraints_inh" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+--------+-----------+----------+---------+---------+-------------+--------------+------------- + c | circle | | | | plain | | | Inherits: test_ex_constraints DROP TABLE test_ex_constraints_inh; @@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY); CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id)); CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints); \d+ test_primary_constraints - Table "public.test_primary_constraints" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - id | integer | | not null | | plain | | + Table "public.test_primary_constraints" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + id | integer | | not null | | plain | | | Indexes: "test_primary_constraints_pkey" PRIMARY KEY, btree (id) Referenced by: TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id) \d+ test_foreign_constraints - Table "public.test_foreign_constraints" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - id1 | integer | | | | plain | | + Table "public.test_foreign_constraints" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + id1 | integer | | | | plain | | | Foreign-key constraints: "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id) Child tables: test_foreign_constraints_inh ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey; \d+ test_foreign_constraints - Table "public.test_foreign_constraints" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - id1 | integer | | | | plain | | + Table "public.test_foreign_constraints" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + id1 | integer | | | | plain | | | Child tables: test_foreign_constraints_inh \d+ test_foreign_constraints_inh - Table "public.test_foreign_constraints_inh" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - id1 | integer | | | | plain | | + Table "public.test_foreign_constraints_inh" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + id1 | integer | | | | plain | | | Inherits: test_foreign_constraints DROP TABLE test_foreign_constraints_inh; diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out index 9d84ba4658..bbab453247 100644 --- a/src/test/regress/expected/insert.out +++ b/src/test/regress/expected/insert.out @@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also insert into inserttest (f4[1].if1, f4[1].if2[2]) select new.f1, new.f2; \d+ inserttest2 - Table "public.inserttest2" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+--------+-----------+----------+---------+----------+--------------+------------- - f1 | bigint | | | | plain | | - f2 | text | | | | extended | | + Table "public.inserttest2" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+--------+-----------+----------+---------+----------+-------------+--------------+------------- + f1 | bigint | | | | plain | | | + f2 | text | | | | extended | | | Rules: irule1 AS ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2]) @@ -435,10 +435,10 @@ drop function dummy_hashint4(a int4, seed int8); create table list_parted (a int) partition by list (a); create table part_default partition of list_parted default; \d+ part_default - Table "public.part_default" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - a | integer | | | | plain | | + Table "public.part_default" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + a | integer | | | | plain | | | Partition of: list_parted DEFAULT No partition constraint @@ -750,74 +750,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE) \d+ mcrparted1_lt_b - Table "public.mcrparted1_lt_b" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted1_lt_b" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text)) \d+ mcrparted2_b - Table "public.mcrparted2_b" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted2_b" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text)) \d+ mcrparted3_c_to_common - Table "public.mcrparted3_c_to_common" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted3_c_to_common" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text)) \d+ mcrparted4_common_lt_0 - Table "public.mcrparted4_common_lt_0" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted4_common_lt_0" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0)) \d+ mcrparted5_common_0_to_10 - Table "public.mcrparted5_common_0_to_10" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted5_common_0_to_10" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10)) \d+ mcrparted6_common_ge_10 - Table "public.mcrparted6_common_ge_10" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted6_common_ge_10" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10)) \d+ mcrparted7_gt_common_lt_d - Table "public.mcrparted7_gt_common_lt_d" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted7_gt_common_lt_d" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text)) \d+ mcrparted8_ge_d - Table "public.mcrparted8_ge_d" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.mcrparted8_ge_d" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE) Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text)) diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out index b101331d69..f805e13d75 100644 --- a/src/test/regress/expected/publication.out +++ b/src/test/regress/expected/publication.out @@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall (1 row) \d+ testpub_tbl2 - Table "public.testpub_tbl2" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | - data | text | | | | extended | | + Table "public.testpub_tbl2" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | | + data | text | | | | extended | | | Indexes: "testpub_tbl2_pkey" PRIMARY KEY, btree (id) Publications: @@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1; ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk; ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1; \d+ pub_test.testpub_nopk - Table "pub_test.testpub_nopk" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - foo | integer | | | | plain | | - bar | integer | | | | plain | | + Table "pub_test.testpub_nopk" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + foo | integer | | | | plain | | | + bar | integer | | | | plain | | | Publications: "testpib_ins_trunct" "testpub_default" "testpub_fortbl" \d+ testpub_tbl1 - Table "public.testpub_tbl1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | - data | text | | | | extended | | + Table "public.testpub_tbl1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | | + data | text | | | | extended | | | Indexes: "testpub_tbl1_pkey" PRIMARY KEY, btree (id) Publications: @@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk; ERROR: relation "testpub_nopk" is not part of the publication \d+ testpub_tbl1 - Table "public.testpub_tbl1" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | - data | text | | | | extended | | + Table "public.testpub_tbl1" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | | + data | text | | | | extended | | | Indexes: "testpub_tbl1_pkey" PRIMARY KEY, btree (id) Publications: diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out index 67c34a92a4..1526437bf8 100644 --- a/src/test/regress/expected/replica_identity.out +++ b/src/test/regress/expected/replica_identity.out @@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass; (1 row) \d+ test_replica_identity - Table "public.test_replica_identity" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+------------- - id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | - keya | text | | not null | | extended | | - keyb | text | | not null | | extended | | - nonkey | text | | | | extended | | + Table "public.test_replica_identity" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+------------- + id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | | + keya | text | | not null | | extended | | | + keyb | text | | not null | | extended | | | + nonkey | text | | | | extended | | | Indexes: "test_replica_identity_pkey" PRIMARY KEY, btree (id) "test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3)) diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index f1c1b44d6f..e358ff539c 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -2799,11 +2799,11 @@ select * from rules_log; create rule r3 as on delete to rules_src do notify rules_src_deletion; \d+ rules_src - Table "public.rules_src" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - f1 | integer | | | | plain | | - f2 | integer | | | | plain | | + Table "public.rules_src" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + f1 | integer | | | | plain | | | + f2 | integer | | | | plain | | | Rules: r1 AS ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text) @@ -2819,11 +2819,11 @@ Rules: create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2; create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1; \d+ rules_src - Table "public.rules_src" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+---------+--------------+------------- - f1 | integer | | | | plain | | - f2 | integer | | | | plain | | + Table "public.rules_src" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+---------+-------------+--------------+------------- + f1 | integer | | | | plain | | | + f2 | integer | | | | plain | | | Rules: r1 AS ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text) diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out index e996640593..62b06da011 100644 --- a/src/test/regress/expected/sanity_check.out +++ b/src/test/regress/expected/sanity_check.out @@ -111,6 +111,8 @@ pg_authid|t pg_cast|t pg_class|t pg_collation|t +pg_compression|t +pg_compression_opt|t pg_constraint|t pg_conversion|t pg_database|t diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out index a4fe96112e..adbe764196 100644 --- a/src/test/regress/expected/update.out +++ b/src/test/regress/expected/update.out @@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10; -- Creating default partition for range create table part_def partition of range_parted default; \d+ part_def - Table "public.part_def" - Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ---------+---------+-----------+----------+---------+----------+--------------+------------- - a | text | | | | extended | | - b | integer | | | | plain | | + Table "public.part_def" + Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description +--------+---------+-----------+----------+---------+----------+-------------+--------------+------------- + a | text | | | | extended | | | + b | integer | | | | plain | | | Partition of: range_parted DEFAULT Partition constraint: (NOT (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20)))) diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index aa5e6af621..a44cf1c910 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta # geometry depends on point, lseg, box, path, polygon and circle # horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime # ---------- -test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions +test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm # ---------- # These four each depend on the previous one diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule index 3866314a92..5c72cb16f5 100644 --- a/src/test/regress/serial_schedule +++ b/src/test/regress/serial_schedule @@ -43,6 +43,7 @@ test: inet test: macaddr test: macaddr8 test: tstypes +test: create_cm test: geometry test: horology test: regex diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql new file mode 100644 index 0000000000..f4701a1dbf --- /dev/null +++ b/src/test/regress/sql/create_cm.sql @@ -0,0 +1,58 @@ +-- test drop +CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler; +CREATE TABLE droptest(fts tsvector COMPRESSED ts1); +DROP COMPRESSION METHOD ts1; +DROP COMPRESSION METHOD ts1 CASCADE; +\d+ droptest +DROP TABLE droptest; + +CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler; +CREATE TABLE cmtest(fts tsvector COMPRESSED ts1); +SELECT * FROM pg_compression; +SELECT cmhandler, cmoptions FROM pg_compression_opt; + +\dCM +\d+ cmtest + +INSERT INTO cmtest + SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' ')) + FROM generate_series(1,200) i; + +SELECT length(fts) FROM cmtest; +SELECT length(fts) FROM cmtest; + +-- check ALTER commands +ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED; +\d+ cmtest +ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz'); +ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1; +\d+ cmtest + +-- create different types of tables +SELECT * INTO cmtest2 FROM cmtest; +CREATE TABLE cmtest3 (LIKE cmtest); +CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest); +CREATE TABLE cmtest5(fts tsvector); + +-- we update usual datum with compressed datum +INSERT INTO cmtest5 + SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' ')) + FROM generate_series(1,200) i; +UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest; + +\d+ cmtest3 +\d+ cmtest4 +DROP TABLE cmtest CASCADE; +DROP TABLE cmtest3; + +SELECT cmhandler, cmoptions FROM pg_compression_opt; + +DROP COMPRESSION METHOD ts1 CASCADE; +SELECT * FROM pg_compression; +SELECT * FROM pg_compression_opt; + +-- check that moved tuples still can be decompressed +SELECT length(fts) FROM cmtest2; +SELECT length(fts) FROM cmtest5; +DROP TABLE cmtest2; +DROP TABLE cmtest5; --MP_/QRJJj=Wu9Xyc77uyK8Zsh.S--