($INBOX_DIR/description missing)
help / color / mirror / Atom feed[PATCH v24 06/10] alter table set compression
123+ messages / 5 participants
[nested] [flat]
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 07/12] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7adeaedd0e..5e24af9cb3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b64db82f02..2aedddc1d8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2107,7 +2107,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--9Ek0hoCL9XbhcSqy
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="0008-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 06/10] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--m51xatjYGsM+13rf
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v21 3/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 ++++++++++++++++----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++
src/test/regress/expected/compression_1.out | 57 ++++++
src/test/regress/sql/compression.sql | 21 ++
9 files changed, 329 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..c4b53ec1c5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. The Compression method can be
+ set to any available compression method. The built-in methods are
+ <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 082db26bfa..89cca760c5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5527,6 +5535,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7767,6 +7778,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7858,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7921,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15069,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 355e273192..f2e9fe4ff8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2307,6 +2307,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 17f7265038..7aeabf9543 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 232c9f1248..ffb739f565 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,7 +1875,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4dea269fc3..84d31c781b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -142,6 +142,65 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index a50c7679bb..3250c12859 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -139,6 +139,63 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0750a6c80e..3b0da88c19 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -65,6 +65,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--ytoMbUMiTKPMT3hY
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v21-0004-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH v24 6/7] alter table set compression
@ 2020-12-18 06:01 dilipkumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: dilipkumar @ 2020-12-18 06:01 UTC (permalink / raw)
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--YZ5djTAD1cGYuMQK
Content-Type: text/x-diff; charset=iso-8859-1
Content-Disposition: attachment;
filename="v24-0007-Add-support-for-PRESERVE.patch"
Content-Transfer-Encoding: 8bit
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--C94crkcyjafcjHxo
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 4/5] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--nHwqXXcoX0o6fKCv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0005-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* [PATCH 7/8] Alter table set compression
@ 2021-03-10 09:04 Dilip Kumar <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Dilip Kumar @ 2021-03-10 09:04 UTC (permalink / raw)
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/[email protected]
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40...
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail...
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail...
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
--cvVnyQ+4j833TQvp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0008-default-to-with-lz4.patch"
^ permalink raw reply [nested|flat] 123+ messages in thread
* Re: pg_upgrade failing for 200+ million Large Objects
@ 2023-11-09 17:35 Kumar, Sachin <[email protected]>
0 siblings, 1 reply; 123+ messages in thread
From: Kumar, Sachin @ 2023-11-09 17:35 UTC (permalink / raw)
To: Jacob Champion <[email protected]>; Nathan Bossart <[email protected]>; +Cc: Jan Wieck <[email protected]>; Tom Lane <[email protected]>; Bruce Momjian <[email protected]>; Zhihong Yu <[email protected]>; Andrew Dunstan <[email protected]>; Magnus Hagander <[email protected]>; Robins Tharakan <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers

Hi Everyone , I want to continue this thread , I have rebased the patch to latest
master and fixed an issue when pg_restore prints to file.
`
╰─$ pg_restore dump_small.custom --restore-blob-batch-size=2 --file=a
--
-- End BLOB restore batch
--
COMMIT;
`
> On 09/11/2023, 17:05, "Jacob Champion" <[email protected] <mailto:[email protected]>> wrote:
> To clarify, I agree that pg_dump should contain the core fix. What I'm
> questioning is the addition of --dump-options to make use of that fix
> from pg_upgrade, since it also lets the user do "exciting" new things
> like --exclude-schema and --include-foreign-data and so on. I don't
> think we should let them do that without a good reason.
Earlier idea was to not expose these options to users and use [1]
--restore-jobs=NUM --jobs parameter passed to pg_restore
--restore-blob-batch-size=NUM number of blobs restored in one xact
But this was later expanded to use --dump-options and --restore-options [2].
With --restore-options user can use --exclude-schema ,
So maybe we can go back to [1]
[1] https://www.postgresql.org/message-id/a1e200e6-adde-2561-422b-a166ec084e3b%40wi3ck.info
[2] https://www.postgresql.org/message-id/8d8d3961-8e8b-3dbe-f911-6f418c5fb1d3%40wi3ck.info
Regards
Sachin
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 123+ messages in thread
* Re: pg_upgrade failing for 200+ million Large Objects
@ 2023-11-09 18:41 Tom Lane <[email protected]>
parent: Kumar, Sachin <[email protected]>
0 siblings, 2 replies; 123+ messages in thread
From: Tom Lane @ 2023-11-09 18:41 UTC (permalink / raw)
To: Kumar, Sachin <[email protected]>; +Cc: Jacob Champion <[email protected]>; Nathan Bossart <[email protected]>; Jan Wieck <[email protected]>; Bruce Momjian <[email protected]>; Zhihong Yu <[email protected]>; Andrew Dunstan <[email protected]>; Magnus Hagander <[email protected]>; Robins Tharakan <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
[ Jacob's email address updated ]
"Kumar, Sachin" <[email protected]> writes:
> Hi Everyone , I want to continue this thread , I have rebased the patch to latest
> master and fixed an issue when pg_restore prints to file.
Um ... you didn't attach the patch?
FWIW, I agree with Jacob's concern about it being a bad idea to let
users of pg_upgrade pass down arbitrary options to pg_dump/pg_restore.
I think we'd regret going there, because it'd hugely expand the set
of cases pg_upgrade has to deal with.
Also, pg_upgrade is often invoked indirectly via scripts, so I do
not especially buy the idea that we're going to get useful control
input from some human somewhere. I think we'd be better off to
assume that pg_upgrade is on its own to manage the process, so that
if we need to switch strategies based on object count or whatever,
we should put in a heuristic to choose the strategy automatically.
It might not be perfect, but that will give better results for the
pretty large fraction of users who are not going to mess with
weird little switches.
regards, tom lane
^ permalink raw reply [nested|flat] 123+ messages in thread
* Re: pg_upgrade failing for 200+ million Large Objects
@ 2023-11-09 23:12 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 0 replies; 123+ messages in thread
From: Andres Freund @ 2023-11-09 23:12 UTC (permalink / raw)
To: [email protected]; Tom Lane <[email protected]>; Kumar, Sachin <[email protected]>; +Cc: Jacob Champion <[email protected]>; Nathan Bossart <[email protected]>; Jan Wieck <[email protected]>; Bruce Momjian <[email protected]>; Zhihong Yu <[email protected]>; Andrew Dunstan <[email protected]>; Magnus Hagander <[email protected]>; Robins Tharakan <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Hi,
On November 9, 2023 10:41:01 AM PST, Tom Lane <[email protected]> wrote:
>Also, pg_upgrade is often invoked indirectly via scripts, so I do
>not especially buy the idea that we're going to get useful control
>input from some human somewhere. I think we'd be better off to
>assume that pg_upgrade is on its own to manage the process, so that
>if we need to switch strategies based on object count or whatever,
>we should put in a heuristic to choose the strategy automatically.
>It might not be perfect, but that will give better results for the
>pretty large fraction of users who are not going to mess with
>weird little switches.
+1 - even leaving everything else aside, just about no user would know about the option. There are cases where we can't do better than giving the user control, but we are certainly adding options at a rate that doesn't seem sustainable. And here it doesn't seem that hard to do better.
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
^ permalink raw reply [nested|flat] 123+ messages in thread
* Re: pg_upgrade failing for 200+ million Large Objects
@ 2023-12-04 16:07 Kumar, Sachin <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 123+ messages in thread
From: Kumar, Sachin @ 2023-12-04 16:07 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jacob Champion <[email protected]>; Nathan Bossart <[email protected]>; Jan Wieck <[email protected]>; Bruce Momjian <[email protected]>; Zhihong Yu <[email protected]>; Andrew Dunstan <[email protected]>; Magnus Hagander <[email protected]>; Robins Tharakan <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
> "Tom Lane" <[email protected] <mailto:[email protected]>> wrote:
> FWIW, I agree with Jacob's concern about it being a bad idea to let
> users of pg_upgrade pass down arbitrary options to pg_dump/pg_restore.
> I think we'd regret going there, because it'd hugely expand the set
> of cases pg_upgrade has to deal with.
> Also, pg_upgrade is often invoked indirectly via scripts, so I do
> not especially buy the idea that we're going to get useful control
> input from some human somewhere. I think we'd be better off to
> assume that pg_upgrade is on its own to manage the process, so that
> if we need to switch strategies based on object count or whatever,
> we should put in a heuristic to choose the strategy automatically.
> It might not be perfect, but that will give better results for the
> pretty large fraction of users who are not going to mess with
> weird little switches.
I have updated the patch to use heuristic, During pg_upgrade we count
Large objects per database. During pg_restore execution if db large_objects
count is greater than LARGE_OBJECTS_THRESOLD (1k) we will use
--restore-blob-batch-size.
I also modified pg_upgrade --jobs behavior if we have large_objects (> LARGE_OBJECTS_THRESOLD)
+ /* Restore all the dbs where LARGE_OBJECTS_THRESOLD is not breached */
+ restore_dbs(stats, true);
+ /* reap all children */
+ while (reap_child(true) == true)
+ ;
+ /* Restore rest of the dbs one by one with pg_restore --jobs = user_opts.jobs */
+ restore_dbs(stats, false);
/* reap all children */
while (reap_child(true) == true)
;
Regards
Sachin
Attachments:
[application/octet-stream] pg_upgrade_improvements_v7.diff (27.9K, ../../[email protected]/2-pg_upgrade_improvements_v7.diff)
download | inline diff:
diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c
index 85e6515ac2..9328830b2e 100644
--- a/src/bin/pg_dump/parallel.c
+++ b/src/bin/pg_dump/parallel.c
@@ -858,6 +858,11 @@ RunWorker(ArchiveHandle *AH, ParallelSlot *slot)
*/
WaitForCommands(AH, pipefd);
+ /*
+ * Close an eventually open BLOB batch transaction.
+ */
+ CommitBlobTransaction((Archive *)AH);
+
/*
* Disconnect from database and clean up.
*/
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9ef2f2017e..65519791e9 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -223,6 +223,8 @@ typedef struct Archive
int numWorkers; /* number of parallel processes */
char *sync_snapshot_id; /* sync snapshot id for parallel operation */
+ int blobBatchSize; /* # of blobs to restore per transaction */
+
/* info needed for string escaping */
int encoding; /* libpq code for client_encoding */
bool std_strings; /* standard_conforming_strings */
@@ -293,6 +295,7 @@ extern void WriteData(Archive *AHX, const void *data, size_t dLen);
extern int StartLO(Archive *AHX, Oid oid);
extern int EndLO(Archive *AHX, Oid oid);
+extern void CommitBlobTransaction(Archive *AH);
extern void CloseArchive(Archive *AHX);
extern void SetArchiveOptions(Archive *AH, DumpOptions *dopt, RestoreOptions *ropt);
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 256d1e35a4..43be945064 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -45,6 +45,7 @@
#define TEXT_DUMP_HEADER "--\n-- PostgreSQL database dump\n--\n\n"
#define TEXT_DUMPALL_HEADER "--\n-- PostgreSQL database cluster dump\n--\n\n"
+static int blobBatchCount = 0;
static ArchiveHandle *_allocAH(const char *FileSpec, const ArchiveFormat fmt,
const pg_compress_specification compression_spec,
@@ -258,6 +259,23 @@ CloseArchive(Archive *AHX)
pg_fatal("could not close output file: %m");
}
+/* Public */
+void
+CommitBlobTransaction(Archive *AHX)
+{
+ ArchiveHandle *AH = (ArchiveHandle *) AHX;
+
+ if (blobBatchCount > 0)
+ {
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "-- End BLOB restore batch\n");
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "COMMIT;\n\n");
+
+ blobBatchCount = 0;
+ }
+}
+
/* Public */
void
SetArchiveOptions(Archive *AH, DumpOptions *dopt, RestoreOptions *ropt)
@@ -719,6 +737,8 @@ RestoreArchive(Archive *AHX)
ahprintf(AH, "COMMIT;\n\n");
}
+ CommitBlobTransaction(AHX);
+
if (AH->public.verbose)
dumpTimestamp(AH, "Completed on", time(NULL));
@@ -3543,6 +3563,57 @@ _printTocEntry(ArchiveHandle *AH, TocEntry *te, bool isData)
{
RestoreOptions *ropt = AH->public.ropt;
+ /* We restore BLOBs in batches to reduce XID consumption */
+ if (strcmp(te->desc, "BLOB") == 0 && AH->public.blobBatchSize > 1)
+ {
+ if (blobBatchCount > 0)
+ {
+ /* We are inside a BLOB restore transaction */
+ if (blobBatchCount >= AH->public.blobBatchSize)
+ {
+ /*
+ * We did reach the batch size with the previous BLOB.
+ * Commit and start a new batch.
+ */
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "-- BLOB batch size reached\n");
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "COMMIT;\n");
+ ahprintf(AH, "BEGIN;\n\n");
+
+ blobBatchCount = 1;
+ }
+ else
+ {
+ /* This one still fits into the current batch */
+ blobBatchCount++;
+ }
+ }
+ else
+ {
+ /* Not inside a transaction, start a new batch */
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "-- Start BLOB restore batch\n");
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "BEGIN;\n\n");
+
+ blobBatchCount = 1;
+ }
+ }
+ else
+ {
+ /* Not a BLOB. If we have a BLOB batch open, close it. */
+ if (blobBatchCount > 0)
+ {
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "-- End BLOB restore batch\n");
+ ahprintf(AH, "--\n");
+ ahprintf(AH, "COMMIT;\n\n");
+
+ blobBatchCount = 0;
+ }
+ }
+
/* Select owner, schema, tablespace and default AM as necessary */
_becomeOwner(AH, te);
_selectOutputSchema(AH, te->namespace);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e863913849..2c6d49732b 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -205,11 +205,20 @@ static inline void dumpComment(Archive *fout, const char *type,
const char *name, const char *namespace,
const char *owner, CatalogId catalogId,
int subid, DumpId dumpId);
+static bool dumpCommentQuery(Archive *fout, PQExpBuffer query, PQExpBuffer tag,
+ const char *type, const char *name,
+ const char *namespace, const char *owner,
+ CatalogId catalogId, int subid, DumpId dumpId,
+ const char *initdb_comment);
static int findComments(Oid classoid, Oid objoid, CommentItem **items);
static void collectComments(Archive *fout);
static void dumpSecLabel(Archive *fout, const char *type, const char *name,
const char *namespace, const char *owner,
CatalogId catalogId, int subid, DumpId dumpId);
+static bool dumpSecLabelQuery(Archive *fout, PQExpBuffer query, PQExpBuffer tag,
+ const char *type, const char *name,
+ const char *namespace, const char *owner,
+ CatalogId catalogId, int subid, DumpId dumpId);
static int findSecLabels(Oid classoid, Oid objoid, SecLabelItem **items);
static void collectSecLabels(Archive *fout);
static void dumpDumpableObject(Archive *fout, DumpableObject *dobj);
@@ -265,6 +274,12 @@ static DumpId dumpACL(Archive *fout, DumpId objDumpId, DumpId altDumpId,
const char *type, const char *name, const char *subname,
const char *nspname, const char *owner,
const DumpableAcl *dacl);
+static bool dumpACLQuery(Archive *fout, PQExpBuffer query, PQExpBuffer tag,
+ DumpId objDumpId, DumpId altDumpId,
+ const char *type, const char *name,
+ const char *subname,
+ const char *nspname, const char *owner,
+ const DumpableAcl *dacl);
static void getDependencies(Archive *fout);
static void BuildArchiveDependencies(Archive *fout);
@@ -3641,10 +3656,42 @@ dumpLO(Archive *fout, const LoInfo *loinfo)
{
PQExpBuffer cquery = createPQExpBuffer();
PQExpBuffer dquery = createPQExpBuffer();
+ PQExpBuffer tag = createPQExpBuffer();
+ teSection section = SECTION_PRE_DATA;
appendPQExpBuffer(cquery,
"SELECT pg_catalog.lo_create('%s');\n",
loinfo->dobj.name);
+ /*
+ * In binary upgrade mode we put all the queries to restore
+ * one large object into a single TOC entry and emit it as
+ * SECTION_DATA so that they can be restored in parallel.
+ */
+ if (fout->dopt->binary_upgrade)
+ {
+ section = SECTION_DATA;
+
+ /* Dump comment if any */
+ if (loinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
+ dumpCommentQuery(fout, cquery, tag, "LARGE OBJECT",
+ loinfo->dobj.name, NULL, loinfo->rolname,
+ loinfo->dobj.catId, 0, loinfo->dobj.dumpId, NULL);
+
+ /* Dump security label if any */
+ if (loinfo->dobj.dump & DUMP_COMPONENT_SECLABEL)
+ dumpSecLabelQuery(fout, cquery, tag, "LARGE OBJECT",
+ loinfo->dobj.name,
+ NULL, loinfo->rolname,
+ loinfo->dobj.catId, 0, loinfo->dobj.dumpId);
+
+ /* Dump ACL if any */
+ if (loinfo->dobj.dump & DUMP_COMPONENT_ACL)
+ dumpACLQuery(fout, cquery, tag,
+ loinfo->dobj.dumpId, InvalidDumpId, "LARGE OBJECT",
+ loinfo->dobj.name, NULL,
+ NULL, loinfo->rolname, &loinfo->dacl);
+ }
+
appendPQExpBuffer(dquery,
"SELECT pg_catalog.lo_unlink('%s');\n",
@@ -3655,27 +3702,28 @@ dumpLO(Archive *fout, const LoInfo *loinfo)
ARCHIVE_OPTS(.tag = loinfo->dobj.name,
.owner = loinfo->rolname,
.description = "BLOB",
- .section = SECTION_PRE_DATA,
+ .section = section,
.createStmt = cquery->data,
.dropStmt = dquery->data));
- /* Dump comment if any */
- if (loinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
- dumpComment(fout, "LARGE OBJECT", loinfo->dobj.name,
- NULL, loinfo->rolname,
- loinfo->dobj.catId, 0, loinfo->dobj.dumpId);
-
- /* Dump security label if any */
- if (loinfo->dobj.dump & DUMP_COMPONENT_SECLABEL)
- dumpSecLabel(fout, "LARGE OBJECT", loinfo->dobj.name,
- NULL, loinfo->rolname,
- loinfo->dobj.catId, 0, loinfo->dobj.dumpId);
-
- /* Dump ACL if any */
- if (loinfo->dobj.dump & DUMP_COMPONENT_ACL)
- dumpACL(fout, loinfo->dobj.dumpId, InvalidDumpId, "LARGE OBJECT",
- loinfo->dobj.name, NULL,
- NULL, loinfo->rolname, &loinfo->dacl);
+ if (!fout->dopt->binary_upgrade)
+ {
+ /* Dump comment if any */
+ if (loinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
+ dumpComment(fout, "LARGE OBJECT", loinfo->dobj.name,
+ NULL, loinfo->rolname,
+ loinfo->dobj.catId, 0, loinfo->dobj.dumpId);
+ /* Dump security label if any */
+ if (loinfo->dobj.dump & DUMP_COMPONENT_SECLABEL)
+ dumpSecLabel(fout, "LARGE OBJECT", loinfo->dobj.name,
+ NULL, loinfo->rolname,
+ loinfo->dobj.catId, 0, loinfo->dobj.dumpId);
+ /* Dump ACL if any */
+ if (loinfo->dobj.dump & DUMP_COMPONENT_ACL)
+ dumpACL(fout, loinfo->dobj.dumpId, InvalidDumpId, "LARGE OBJECT",
+ loinfo->dobj.name, NULL,
+ NULL, loinfo->rolname, &loinfo->dacl);
+ }
destroyPQExpBuffer(cquery);
destroyPQExpBuffer(dquery);
@@ -9899,6 +9947,38 @@ dumpCommentExtended(Archive *fout, const char *type,
const char *owner, CatalogId catalogId,
int subid, DumpId dumpId,
const char *initdb_comment)
+{
+ PQExpBuffer query = createPQExpBuffer();
+ PQExpBuffer tag = createPQExpBuffer();
+
+ if (dumpCommentQuery(fout, query, tag, type, name, namespace, owner,
+ catalogId, subid, dumpId, initdb_comment))
+ {
+ /*
+ * We mark comments as SECTION_NONE because they really belong in the
+ * same section as their parent, whether that is pre-data or
+ * post-data.
+ */
+ ArchiveEntry(fout, nilCatalogId, createDumpId(),
+ ARCHIVE_OPTS(.tag = tag->data,
+ .namespace = namespace,
+ .owner = owner,
+ .description = "COMMENT",
+ .section = SECTION_NONE,
+ .createStmt = query->data,
+ .deps = &dumpId,
+ .nDeps = 1));
+ }
+ destroyPQExpBuffer(query);
+ destroyPQExpBuffer(tag);
+}
+
+static bool
+dumpCommentQuery(Archive *fout, PQExpBuffer query, PQExpBuffer tag,
+ const char *type, const char *name,
+ const char *namespace, const char *owner,
+ CatalogId catalogId, int subid, DumpId dumpId,
+ const char *initdb_comment)
{
DumpOptions *dopt = fout->dopt;
CommentItem *comments;
@@ -9906,19 +9986,19 @@ dumpCommentExtended(Archive *fout, const char *type,
/* do nothing, if --no-comments is supplied */
if (dopt->no_comments)
- return;
+ return false;
/* Comments are schema not data ... except LO comments are data */
if (strcmp(type, "LARGE OBJECT") != 0)
{
if (dopt->dataOnly)
- return;
+ return false;
}
else
{
/* We do dump LO comments in binary-upgrade mode */
if (dopt->schemaOnly && !dopt->binary_upgrade)
- return;
+ return false;
}
/* Search for comments associated with catalogId, using table */
@@ -9956,9 +10036,6 @@ dumpCommentExtended(Archive *fout, const char *type,
/* If a comment exists, build COMMENT ON statement */
if (ncomments > 0)
{
- PQExpBuffer query = createPQExpBuffer();
- PQExpBuffer tag = createPQExpBuffer();
-
appendPQExpBuffer(query, "COMMENT ON %s ", type);
if (namespace && *namespace)
appendPQExpBuffer(query, "%s.", fmtId(namespace));
@@ -9968,24 +10045,10 @@ dumpCommentExtended(Archive *fout, const char *type,
appendPQExpBuffer(tag, "%s %s", type, name);
- /*
- * We mark comments as SECTION_NONE because they really belong in the
- * same section as their parent, whether that is pre-data or
- * post-data.
- */
- ArchiveEntry(fout, nilCatalogId, createDumpId(),
- ARCHIVE_OPTS(.tag = tag->data,
- .namespace = namespace,
- .owner = owner,
- .description = "COMMENT",
- .section = SECTION_NONE,
- .createStmt = query->data,
- .deps = &dumpId,
- .nDeps = 1));
-
- destroyPQExpBuffer(query);
- destroyPQExpBuffer(tag);
+ return true;
}
+
+ return false;
}
/*
@@ -14939,23 +15002,65 @@ dumpACL(Archive *fout, DumpId objDumpId, DumpId altDumpId,
const DumpableAcl *dacl)
{
DumpId aclDumpId = InvalidDumpId;
+ PQExpBuffer query = createPQExpBuffer();
+ PQExpBuffer tag = createPQExpBuffer();
+
+ if (dumpACLQuery(fout, query, tag, objDumpId, altDumpId,
+ type, name, subname, nspname, owner, dacl))
+ {
+ DumpId aclDeps[2];
+ int nDeps = 0;
+
+ if (subname)
+ appendPQExpBuffer(tag, "COLUMN %s.%s", name, subname);
+ else
+ appendPQExpBuffer(tag, "%s %s", type, name);
+
+ aclDeps[nDeps++] = objDumpId;
+ if (altDumpId != InvalidDumpId)
+ aclDeps[nDeps++] = altDumpId;
+
+ aclDumpId = createDumpId();
+
+ ArchiveEntry(fout, nilCatalogId, aclDumpId,
+ ARCHIVE_OPTS(.tag = tag->data,
+ .namespace = nspname,
+ .owner = owner,
+ .description = "ACL",
+ .section = SECTION_NONE,
+ .createStmt = query->data,
+ .deps = aclDeps,
+ .nDeps = nDeps));
+
+ }
+
+ destroyPQExpBuffer(query);
+ destroyPQExpBuffer(tag);
+
+ return aclDumpId;
+}
+
+static bool
+dumpACLQuery(Archive *fout, PQExpBuffer query, PQExpBuffer tag,
+ DumpId objDumpId, DumpId altDumpId,
+ const char *type, const char *name, const char *subname,
+ const char *nspname, const char *owner,
+ const DumpableAcl *dacl)
+{
DumpOptions *dopt = fout->dopt;
const char *acls = dacl->acl;
const char *acldefault = dacl->acldefault;
char privtype = dacl->privtype;
const char *initprivs = dacl->initprivs;
const char *baseacls;
- PQExpBuffer sql;
/* Do nothing if ACL dump is not enabled */
if (dopt->aclsSkip)
- return InvalidDumpId;
+ return false;
/* --data-only skips ACLs *except* large object ACLs */
if (dopt->dataOnly && strcmp(type, "LARGE OBJECT") != 0)
- return InvalidDumpId;
-
- sql = createPQExpBuffer();
+ return false;
/*
* In binary upgrade mode, we don't run an extension's script but instead
@@ -14973,13 +15078,13 @@ dumpACL(Archive *fout, DumpId objDumpId, DumpId altDumpId,
if (dopt->binary_upgrade && privtype == 'e' &&
initprivs && *initprivs != '\0')
{
- appendPQExpBufferStr(sql, "SELECT pg_catalog.binary_upgrade_set_record_init_privs(true);\n");
+ appendPQExpBufferStr(query, "SELECT pg_catalog.binary_upgrade_set_record_init_privs(true);\n");
if (!buildACLCommands(name, subname, nspname, type,
initprivs, acldefault, owner,
- "", fout->remoteVersion, sql))
+ "", fout->remoteVersion, query))
pg_fatal("could not parse initial ACL list (%s) or default (%s) for object \"%s\" (%s)",
initprivs, acldefault, name, type);
- appendPQExpBufferStr(sql, "SELECT pg_catalog.binary_upgrade_set_record_init_privs(false);\n");
+ appendPQExpBufferStr(query, "SELECT pg_catalog.binary_upgrade_set_record_init_privs(false);\n");
}
/*
@@ -15001,43 +15106,19 @@ dumpACL(Archive *fout, DumpId objDumpId, DumpId altDumpId,
if (!buildACLCommands(name, subname, nspname, type,
acls, baseacls, owner,
- "", fout->remoteVersion, sql))
+ "", fout->remoteVersion, query))
pg_fatal("could not parse ACL list (%s) or default (%s) for object \"%s\" (%s)",
acls, baseacls, name, type);
- if (sql->len > 0)
+ if (query->len > 0 && tag != NULL)
{
- PQExpBuffer tag = createPQExpBuffer();
- DumpId aclDeps[2];
- int nDeps = 0;
-
if (subname)
appendPQExpBuffer(tag, "COLUMN %s.%s", name, subname);
else
appendPQExpBuffer(tag, "%s %s", type, name);
-
- aclDeps[nDeps++] = objDumpId;
- if (altDumpId != InvalidDumpId)
- aclDeps[nDeps++] = altDumpId;
-
- aclDumpId = createDumpId();
-
- ArchiveEntry(fout, nilCatalogId, aclDumpId,
- ARCHIVE_OPTS(.tag = tag->data,
- .namespace = nspname,
- .owner = owner,
- .description = "ACL",
- .section = SECTION_NONE,
- .createStmt = sql->data,
- .deps = aclDeps,
- .nDeps = nDeps));
-
- destroyPQExpBuffer(tag);
}
- destroyPQExpBuffer(sql);
-
- return aclDumpId;
+ return true;
}
/*
@@ -15062,16 +15143,42 @@ static void
dumpSecLabel(Archive *fout, const char *type, const char *name,
const char *namespace, const char *owner,
CatalogId catalogId, int subid, DumpId dumpId)
+{
+ PQExpBuffer query = createPQExpBuffer();
+ PQExpBuffer tag = createPQExpBuffer();
+
+ if (dumpSecLabelQuery(fout, query, tag, type, name,
+ namespace, owner, catalogId, subid, dumpId))
+ {
+ ArchiveEntry(fout, nilCatalogId, createDumpId(),
+ ARCHIVE_OPTS(.tag = tag->data,
+ .namespace = namespace,
+ .owner = owner,
+ .description = "SECURITY LABEL",
+ .section = SECTION_NONE,
+ .createStmt = query->data,
+ .deps = &dumpId,
+ .nDeps = 1));
+ }
+
+ destroyPQExpBuffer(query);
+ destroyPQExpBuffer(tag);
+}
+
+static bool
+dumpSecLabelQuery(Archive *fout, PQExpBuffer query, PQExpBuffer tag,
+ const char *type, const char *name,
+ const char *namespace, const char *owner,
+ CatalogId catalogId, int subid, DumpId dumpId)
{
DumpOptions *dopt = fout->dopt;
SecLabelItem *labels;
int nlabels;
int i;
- PQExpBuffer query;
/* do nothing, if --no-security-labels is supplied */
if (dopt->no_security_labels)
- return;
+ return false;
/*
* Security labels are schema not data ... except large object labels are
@@ -15080,20 +15187,18 @@ dumpSecLabel(Archive *fout, const char *type, const char *name,
if (strcmp(type, "LARGE OBJECT") != 0)
{
if (dopt->dataOnly)
- return;
+ return false;
}
else
{
/* We do dump large object security labels in binary-upgrade mode */
if (dopt->schemaOnly && !dopt->binary_upgrade)
- return;
+ return false;
}
/* Search for security labels associated with catalogId, using table */
nlabels = findSecLabels(catalogId.tableoid, catalogId.oid, &labels);
- query = createPQExpBuffer();
-
for (i = 0; i < nlabels; i++)
{
/*
@@ -15114,22 +15219,11 @@ dumpSecLabel(Archive *fout, const char *type, const char *name,
if (query->len > 0)
{
- PQExpBuffer tag = createPQExpBuffer();
-
appendPQExpBuffer(tag, "%s %s", type, name);
- ArchiveEntry(fout, nilCatalogId, createDumpId(),
- ARCHIVE_OPTS(.tag = tag->data,
- .namespace = namespace,
- .owner = owner,
- .description = "SECURITY LABEL",
- .section = SECTION_NONE,
- .createStmt = query->data,
- .deps = &dumpId,
- .nDeps = 1));
- destroyPQExpBuffer(tag);
+ return true;
}
- destroyPQExpBuffer(query);
+ return false;
}
/*
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 049a100634..2159f72ffb 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -60,6 +60,7 @@ main(int argc, char **argv)
int c;
int exit_code;
int numWorkers = 1;
+ int blobBatchSize = 0;
Archive *AH;
char *inputFileSpec;
static int disable_triggers = 0;
@@ -123,6 +124,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"restore-blob-batch-size", required_argument, NULL, 4},
{NULL, 0, NULL, 0}
};
@@ -286,6 +288,10 @@ main(int argc, char **argv)
set_dump_section(optarg, &(opts->dumpSections));
break;
+ case 4: /* # of blobs to restore per transaction */
+ blobBatchSize = atoi(optarg);
+ break;
+
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -405,6 +411,7 @@ main(int argc, char **argv)
SortTocFromFile(AH);
AH->numWorkers = numWorkers;
+ AH->blobBatchSize = blobBatchSize;
if (opts->tocSummary)
PrintTOCSummary(AH);
@@ -478,6 +485,8 @@ usage(const char *progname)
printf(_(" --use-set-session-authorization\n"
" use SET SESSION AUTHORIZATION commands instead of\n"
" ALTER OWNER commands to set ownership\n"));
+ printf(_(" --restore-blob-batch-size=NUM\n"
+ " attempt to restore NUM large objects per transaction\n"));
printf(_("\nConnection options:\n"));
printf(_(" -h, --host=HOSTNAME database server host or socket directory\n"));
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index fa52aa2c22..459d834ac3 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -84,7 +84,7 @@ output_check_banner(bool live_check)
void
-check_and_dump_old_cluster(bool live_check)
+check_and_dump_old_cluster(bool live_check, DbDumpStats **stats)
{
/* -- OLD -- */
@@ -202,12 +202,36 @@ check_and_dump_old_cluster(bool live_check)
* the old server is running.
*/
if (!user_opts.check)
+ {
+ *stats = collect_db_stats();
generate_old_dump();
+ }
if (!live_check)
stop_postmaster(false);
}
+DbDumpStats* collect_db_stats(void)
+{
+ uint dbnum;
+ DbDumpStats *stats = (DbDumpStats *)pg_malloc(sizeof(DbDumpStats));
+ stats->large_objects = (uint64 *)pg_malloc(sizeof(old_cluster.dbarr.ndbs * sizeof(uint64)));
+ prep_status("Collecting database stats");
+ for (dbnum = 0; dbnum < old_cluster.dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ DbInfo *active_db = &old_cluster.dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(&old_cluster, active_db->db_name);
+
+ res = executeQueryOrDie(conn, "SELECT count(*) from pg_largeobject_metadata");
+ stats->large_objects[dbnum] = atoll(PQgetvalue(res, 0, 0));
+ PQclear(res);
+ PQfinish(conn);
+ }
+ check_ok();
+
+ return stats;
+}
void
check_new_cluster(void)
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 3960af4036..12605200b5 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -54,7 +54,8 @@
static void set_locale_and_encoding(void);
static void prepare_new_cluster(void);
static void prepare_new_globals(void);
-static void create_new_objects(void);
+static void restore_dbs(DbDumpStats *stats, bool parallel_restore);
+static void create_new_objects(DbDumpStats *stats);
static void copy_xact_xlog_xid(void);
static void set_frozenxids(bool minmxid_only);
static void make_outputdirs(char *pgdata);
@@ -82,6 +83,7 @@ main(int argc, char **argv)
{
char *deletion_script_file_name = NULL;
bool live_check = false;
+ DbDumpStats *stats;
/*
* pg_upgrade doesn't currently use common/logging.c, but initialize it
@@ -127,7 +129,7 @@ main(int argc, char **argv)
check_cluster_compatibility(live_check);
- check_and_dump_old_cluster(live_check);
+ check_and_dump_old_cluster(live_check, &stats);
/* -- NEW -- */
@@ -160,7 +162,7 @@ main(int argc, char **argv)
prepare_new_globals();
- create_new_objects();
+ create_new_objects(stats);
stop_postmaster(false);
@@ -508,9 +510,72 @@ prepare_new_globals(void)
check_ok();
}
+static void
+restore_dbs(DbDumpStats *stats, bool parallel_restore)
+{
+ int dbnum;
+ for (dbnum = 0; dbnum < old_cluster.dbarr.ndbs; dbnum++)
+ {
+ char sql_file_name[MAXPGPATH],
+ log_file_name[MAXPGPATH];
+ DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
+ const char *create_opts;
+ int jobs = user_opts.jobs ? user_opts.jobs : 1 ;
+ bool large_objects_thresold_breached = stats && stats->large_objects[dbnum] > LARGE_OBJECTS_THRESOLD;
+
+ /* Skip template1 in this pass */
+ if (strcmp(old_db->db_name, "template1") == 0)
+ continue;
+ /* Skip dbs where LARGE_OBJECTS_THRESOLD is breached and parallel_restore is enabled*/
+ if (large_objects_thresold_breached && parallel_restore)
+ continue;
+
+ pg_log(PG_STATUS, "%s", old_db->db_name);
+ snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
+ snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
+
+ /*
+ * postgres database will already exist in the target installation, so
+ * tell pg_restore to drop and recreate it; otherwise we would fail to
+ * propagate its database-level properties.
+ */
+ if (strcmp(old_db->db_name, "postgres") == 0)
+ create_opts = "--clean --create";
+ else
+ create_opts = "--create";
+
+ if (parallel_restore)
+ parallel_exec_prog(log_file_name,
+ NULL,
+ "\"%s/pg_restore\" %s %s --exit-on-error --verbose "
+ "--dbname template1 \"%s/%s\"",
+ new_cluster.bindir,
+ cluster_conn_opts(&new_cluster),
+ create_opts,
+ log_opts.dumpdir,
+ sql_file_name);
+ else
+ exec_prog(log_file_name,
+ NULL,
+ true,
+ true,
+ "\"%s/pg_restore\" %s %s --exit-on-error --verbose "
+ "--restore-blob-batch-size %d --jobs %d "
+ "--dbname template1 \"%s/%s\"",
+ new_cluster.bindir,
+ cluster_conn_opts(&new_cluster),
+ create_opts,
+ large_objects_thresold_breached ?
+ LARGE_OBJECTS_THRESOLD : 0,
+ large_objects_thresold_breached ?
+ jobs : 1,
+ log_opts.dumpdir,
+ sql_file_name);
+ }
+}
static void
-create_new_objects(void)
+create_new_objects(DbDumpStats *stats)
{
int dbnum;
@@ -557,43 +622,13 @@ create_new_objects(void)
break; /* done once we've processed template1 */
}
-
- for (dbnum = 0; dbnum < old_cluster.dbarr.ndbs; dbnum++)
- {
- char sql_file_name[MAXPGPATH],
- log_file_name[MAXPGPATH];
- DbInfo *old_db = &old_cluster.dbarr.dbs[dbnum];
- const char *create_opts;
-
- /* Skip template1 in this pass */
- if (strcmp(old_db->db_name, "template1") == 0)
- continue;
-
- pg_log(PG_STATUS, "%s", old_db->db_name);
- snprintf(sql_file_name, sizeof(sql_file_name), DB_DUMP_FILE_MASK, old_db->db_oid);
- snprintf(log_file_name, sizeof(log_file_name), DB_DUMP_LOG_FILE_MASK, old_db->db_oid);
-
- /*
- * postgres database will already exist in the target installation, so
- * tell pg_restore to drop and recreate it; otherwise we would fail to
- * propagate its database-level properties.
- */
- if (strcmp(old_db->db_name, "postgres") == 0)
- create_opts = "--clean --create";
- else
- create_opts = "--create";
-
- parallel_exec_prog(log_file_name,
- NULL,
- "\"%s/pg_restore\" %s %s --exit-on-error --verbose "
- "--dbname template1 \"%s/%s\"",
- new_cluster.bindir,
- cluster_conn_opts(&new_cluster),
- create_opts,
- log_opts.dumpdir,
- sql_file_name);
- }
-
+ /* Restore all the dbs where LARGE_OBJECTS_THRESOLD is not breached */
+ restore_dbs(stats, true);
+ /* reap all children */
+ while (reap_child(true) == true)
+ ;
+ /* Restore rest of the dbs one by one with pg_restore --jobs = user_opts.jobs */
+ restore_dbs(stats, false);
/* reap all children */
while (reap_child(true) == true)
;
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index a710f325de..f41063dbc7 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -24,6 +24,8 @@
#define MESSAGE_WIDTH 62
+#define LARGE_OBJECTS_THRESOLD 200 // maybe 10k ?
+
#define GET_MAJOR_VERSION(v) ((v) / 100)
/* contains both global db information and CREATE DATABASE commands */
@@ -347,6 +349,15 @@ typedef struct
ClusterInfo *running_cluster;
} OSInfo;
+/*
+ * Dump stats, will be used by pg_upgrade to efficently run pg_restore
+ */
+
+typedef struct
+{
+ uint64 *large_objects;
+}DbDumpStats;
+
/*
* Global variables
@@ -361,7 +372,7 @@ extern OSInfo os_info;
/* check.c */
void output_check_banner(bool live_check);
-void check_and_dump_old_cluster(bool live_check);
+void check_and_dump_old_cluster(bool live_check, DbDumpStats **stats);
void check_new_cluster(void);
void report_clusters_compatible(void);
void issue_warnings_and_set_wal_level(void);
@@ -369,6 +380,7 @@ void output_completion_banner(char *deletion_script_file_name);
void check_cluster_versions(void);
void check_cluster_compatibility(bool live_check);
void create_script_for_old_cluster_deletion(char **deletion_script_file_name);
+DbDumpStats* collect_db_stats(void);
/* controldata.c */
^ permalink raw reply [nested|flat] 123+ messages in thread
* Re: pg_upgrade failing for 200+ million Large Objects
@ 2023-12-07 14:05 Kumar, Sachin <[email protected]>
parent: Kumar, Sachin <[email protected]>
0 siblings, 0 replies; 123+ messages in thread
From: Kumar, Sachin @ 2023-12-07 14:05 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Nathan Bossart <[email protected]>; Jan Wieck <[email protected]>; Bruce Momjian <[email protected]>; Zhihong Yu <[email protected]>; Andrew Dunstan <[email protected]>; Magnus Hagander <[email protected]>; Robins Tharakan <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
> I have updated the patch to use heuristic, During pg_upgrade we count
> Large objects per database. During pg_restore execution if db large_objects
> count is greater than LARGE_OBJECTS_THRESOLD (1k) we will use
> --restore-blob-batch-size.
I think both SECTION_DATA and SECTION_POST_DATA can be parallelized by pg_restore, So instead of storing
large objects in heuristics, we can store SECTION_DATA + SECTION_POST_DATA.
Regards
Sachin
^ permalink raw reply [nested|flat] 123+ messages in thread
end of thread, other threads:[~2023-12-07 14:05 UTC | newest]
Thread overview: 123+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH 07/12] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v21 3/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 6/7] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2020-12-18 06:01 [PATCH v24 06/10] alter table set compression dilipkumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 7/8] Alter table set compression Dilip Kumar <[email protected]>
2021-03-10 09:04 [PATCH 4/5] Alter table set compression Dilip Kumar <[email protected]>
2023-11-09 17:35 Re: pg_upgrade failing for 200+ million Large Objects Kumar, Sachin <[email protected]>
2023-11-09 18:41 ` Re: pg_upgrade failing for 200+ million Large Objects Tom Lane <[email protected]>
2023-11-09 23:12 ` Re: pg_upgrade failing for 200+ million Large Objects Andres Freund <[email protected]>
2023-12-04 16:07 ` Re: pg_upgrade failing for 200+ million Large Objects Kumar, Sachin <[email protected]>
2023-12-07 14:05 ` Re: pg_upgrade failing for 200+ million Large Objects Kumar, Sachin <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox